Commit Graph

140 Commits

Author SHA1 Message Date
Yoshino-s b57c8c31db feat(flags): add screenshot-timeout flag (#1516)
* chore(deps): bump golang.org/x/net from 0.14.0 to 0.17.0 (#1400)

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.14.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#1493)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.14.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.14.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat(flags): add screenshot-timeout flag

* misc update

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-01-16 04:08:04 +05:30
mzack e557bf8662 using fastdialer deny 2024-01-07 21:10:56 +01:00
Doğan Can Bakır af5909bcac introduce -exclude flag 2024-01-04 15:16:02 +03:00
seeyarh 91139dbc91 Fix chrome unsafe port err by setting explicitly-allowed-ports (#1343)
* Update headless.go

* adding headless generic option

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2023-11-28 18:18:29 +05:30
Cody Cline 7dee9988ca Feature: optional flag to skip a host if it has a private IP address. (#1408)
* feat: Implement flag which when enabled, will skip any host with a private ip address instead of timing out.

* fix: Amend private ip check to include loopback and link-local addresses

* fix: Amend debug message to be more concise

* fix: Account for port numbers in skipPrivateHosts feature

* docs: Remove uncessary  debug statement

* readme update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-11-02 14:21:22 +05:30
Doğan Can Bakır 0f1b51b6c8 introduce visual recon clusters 2023-09-20 08:31:30 +00:00
Mzack9999 88fa321dc4 Restoring https://github.com/projectdiscovery/httpx/pull/1331 (#1360)
* Restoring https://github.com/projectdiscovery/httpx/pull/1331

* updating readme
2023-09-14 17:51:38 +05:30
sandeep d380f3139b readme update 2023-09-13 23:15:18 +05:30
sandeep b6793864ee readme updates 2023-09-12 16:30:03 +05:30
JoMar 11fa743dd9 Allow to show response headers in json output without using -irh option (#1326)
* Show responses headers in JSON output

* Update ident

* Update options name

* Update option & README

* Add missing 's'

* Add missing 's'

* updated headers to header for user facing variable to keep it uniform

* updating doc

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-09-12 03:37:54 +05:30
Mzack9999 0f6edb50be refactor hsts implementation 2023-09-11 17:53:26 +02:00
Mzack9999 8b5dc7bc0f merge conflict 2023-09-11 16:58:31 +02:00
Ramana Reddy 76adadfeb4 Add flag to preview first N characters (#1316)
* Add flag to preview first N characters

* use dynamic var for body-preview flag

* strip html tags from response

* add stripe option to remove html or xml tags

* typo

* resolve merge conflicts

* add test and update readme

* update readme

* typo

* typo no 2

* small refactor

* include body preview in jsonl output

* minor changes

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-09-07 15:26:54 +05:30
Stefan Kraxberger 964cc3bbd0 Implemented a redirect strategy which respects HSTS headers and modifies the URL accordingly if necessary 2023-08-31 10:58:51 +02:00
secinto d463b3bcda Merge branch 'dev' into feature_1330 2023-08-27 16:09:40 +02:00
Stefan Kraxberger 9be9fb59cb Implementation of feature request #1330 2023-08-24 11:42:10 +02:00
DroidKali bf1f88438d runner/options.go: Fixed typo. (#1310)
Signed-off-by: DroidKali <DroidKali@users.noreply.github.com>
Co-authored-by: DroidKali <DroidKali@users.noreply.github.com>
2023-08-04 15:52:07 +05:30
Ivan Novikov 9e1242d7e0 Make output response fields public (#1266)
* Make output response fields public so they can be specified if httpx used as a library.

* Fix runner.go to use renamed params.

* renaming var

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2023-07-19 18:26:53 +05:30
sandeep 366a98493b Merge branch 'dev' into pr/1245 2023-07-17 16:43:04 +05:30
Dogan Can Bakir c3eb8897e7 Support multiple output format (#1242)
* support multiple output format

* output plain text when -oA

* separate the conditions

* fail output-all when it's not used with output

* update doc

* remove hashes for plain output when -hash isn't used

* add integration test

* use filepath

* fix no hash std output

* remove json/csv extension for json/csv only output

* add ext when both -csv and -json is used

* fix assertion

* commit to last commit

* print filenames

* no tmpdir

* remove files

* remove prints

* lowercase options

* test update

---------

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-07-17 16:09:10 +05:30
Doğan Can Bakır 2b1ab2a145 Merge branch 'dev' into error_page_classifier 2023-07-10 08:57:45 +03:00
Dogan Can Bakir 5b21d47879 add config flag (#1273) 2023-07-05 20:04:56 +05:30
Doğan Can Bakır 900f5623e8 add error page classifier 2023-06-20 14:13:48 +03:00
Mzack9999 8ea85e901d Adding prototype of tls impersonate (#1199)
* Adding prototype of tls impersonate

* updating docs

* misc update

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-17 16:34:19 +05:30
Recar 5984aad82f add option no-stdin (#1213) 2023-06-12 03:03:16 +05:30
shubhamrasal bc7ac8d073 Update callback calling place
- Callback for every result, even it contains error
- User can handle it using Result.Err
2023-05-19 17:27:29 +05:30
mzack 318f9cff61 Merge branch 'dev' into sdk-code-improvements 2023-05-06 19:01:40 +02:00
dependabot[bot] 7d41db16f9 chore(deps): bump github.com/projectdiscovery/cdncheck from 0.0.4-0.20220413175814-b47bc2d578b1 to 1.0.0 (#1118)
* chore(deps): bump github.com/projectdiscovery/cdncheck

Bumps [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) from 0.0.4-0.20220413175814-b47bc2d578b1 to 1.0.0.
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Changelog](https://github.com/projectdiscovery/cdncheck/blob/main/.goreleaser.yaml)
- [Commits](https://github.com/projectdiscovery/cdncheck/commits/v1.0.0)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* cdncheck update fixes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ramana Reddy <ramanaredy.manda@gmail.com>
2023-04-27 12:46:28 +05:30
Mzack9999 00d0977963 Adding Screenshot support (#1097)
* Adding Screenshot support

* adding headless body

* moving example into compilable file

* updating docs

* enabling store response with screenshot

* fixing output dir path

* fixing mod

* using native leakless

* splitting response+screenshot folders

* better handling json output

* readme update

* rel => abs path

* Added chromium into docker

* “Wipe them out. All of them.” – Darth Sidious

* utils bump

* go mod tidy

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-27 01:07:06 +05:30
Ramana Reddy be66fc3fd1 update utils pkg (#1109) 2023-04-20 14:17:14 +05:30
Mzack9999 788ef43b1c Merge pull request #1104 from ireland/patch-1
Minor update options.go to fix a typo in -leave-default-ports descrip…
2023-04-19 08:15:50 +02:00
Ireland 64fa729c0c Minor update options.go to fix a typo in -leave-default-ports description text
Help text fix to change "http://host:80 - https//host:443" to "http://host:80 - https://host:443" for consistency
2023-04-18 22:36:28 +01:00
Ice3man 0dc39fe249 Added exported ScanOptions + Raw + Err methods 2023-04-12 20:44:46 +05:30
Mzack9999 1defd0a9b7 Add support to skip decode (#1055)
add support to skip decode
2023-03-29 19:06:47 +05:30
Ramana Reddy 948dd32663 add update check and self update options (#1041)
Add update check and self update option (#946)
2023-03-14 20:37:42 +05:30
xm1k3 88c49b812b added ztls flag (#896)
* added ztls to httpx

* adding missing dialtls

* updated readme.md

* add ztls functional test

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-28 23:40:22 +05:30
xm1k3 b8de450be6 bom utf8 on csv output (#887)
* implemented bom utf8 on csv

* added on csvoutput

* Removed debug code

* added output encoding flag

* using strings utils

* small refactor

* removing default encoding

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2022-11-29 12:18:38 +05:30
Tarun Koyalwar 20a9a930c2 adds base64 encoding support (#813) (#885)
* adds base64 encoding support (#813)

* rename b64 to base64
2022-11-24 21:50:37 +05:30
Sami 45f9f5c199 Use a random hash instead of generating an url-based filename to store (#848)
* Use a random hash instead of generating an url-based filename to store

* switching to sha1 hash generation

* removing max filename limit and reusing existing helper

* hash for filename to store improvements

* lint error fix

* used filepath to be cross platform

* used strings.Builder to prevent continous opening and closing of file

* lint error fix

* refactoring output logic

* fixing lint errors

* Using Sha1 to reduce file length

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-11-20 17:02:24 +05:30
Shubham Rasal f06133f03b Add delay between making httpx requests. (#853)
* Add delay between making httpx requests.

- sleep before creating goroutine to make http request
- eg.
  ```
  cat urls.txt | httpx -delay 2s
  ```
  ```
  cat urls.txt | httpx -delay 2s -port 80,443,8443
  ```

* Update delay help message

* Use goflags DurationVar to accept delay

* replace string with duration for delay
2022-11-15 12:23:45 +05:30
vrenzolaverace 686a2cced8 Update helpers libraries #850 (#851)
Use utils helpers libraries (#850)
2022-11-07 01:52:47 +05:30
mzack 7a9a3fd593 fixing lint errors 2022-10-28 16:24:13 +02:00
Mzack9999 4eb398ed41 Merge branch 'dev' into pr/825 2022-10-24 10:03:40 +02:00
Mzack9999 a90e5a140c minor signature change 2022-10-24 10:00:54 +02:00
lucy9x 7d7807c832 add OnResultCallback options 2022-10-24 13:33:20 +07:00
Sami 12fee4f3ee include dsl tag for all json keys (#812)
* include dsl tag for all json keys

* removed dsl tag and built list from json tag

* asn interface changed

* refactoring expression engine

- Using mapstructure for uniform mapping
- Flat key:value in expressions/variables list

* adding fields to ignore

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-10-20 13:18:01 +05:30
mzack 7f98e13c4a updating goflags syntax 2022-10-12 00:15:04 +02:00
Sajad e31c1250a2 add dsl match/filter condtion (#782)
* add dsl match/filter condtion

* update README

* add test cases
2022-10-06 17:11:12 +05:30
Ismael Arias d48afd2a79 Added flag for single or comma separated stdin input values (#737) (#759)
* Added flag for single or comma separated stdin input values (#737)

* Replaced the logic in the parsing of input target hosts flag.
Using the goflags' StringSlice now.

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-09-21 20:03:36 +05:30
Sami c90b897eba list-dsl flag to list json output keys which can be further used with matchers / filters (#768)
* flag list-dsl

* variable name update

* used reflect to dnymically access DSL variables

* aligned with the changes in dev branch

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-09-19 22:50:40 +05:30