Commit Graph

233 Commits

Author SHA1 Message Date
Dogan Can Bakir b9a444795d Use useragent (#1562)
* use projectdiscovery/useragent

* fix go.mod

* update useragent

* commit to last commit
2024-01-31 20:53:17 +05:30
Mzack9999 7594c8bf5d Merge branch 'dev' into orig-1469 2024-01-15 22:03:56 +01:00
mzack 0061bc3a20 small description change 2024-01-15 21:57:41 +01:00
Ramana Reddy db1ea9fad3 Fix issue with host redirect on ports flag (#1529) 2024-01-15 15:27:24 +05:30
Ramana Reddy 64b6a0789c Fix: Pipeline and Websocket not working for some targets (#1528)
* Fix pipeline issue for port included targets

* Add more checks to websocket
2024-01-15 15:06:28 +05:30
mzack e557bf8662 using fastdialer deny 2024-01-07 21:10:56 +01:00
Karel f3a6eadfce Do not use local resolvers when given resolver input 2023-11-27 16:37:06 +01:00
Doğan Can Bakır d55d459f1d extend char set 2023-11-08 07:54:22 +00:00
Doğan Can Bakır 37591e1dd5 fix -title newline issue 2023-11-07 19:50:28 +00:00
Cody Cline 7dee9988ca Feature: optional flag to skip a host if it has a private IP address. (#1408)
* feat: Implement flag which when enabled, will skip any host with a private ip address instead of timing out.

* fix: Amend private ip check to include loopback and link-local addresses

* fix: Amend debug message to be more concise

* fix: Account for port numbers in skipPrivateHosts feature

* docs: Remove uncessary  debug statement

* readme update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-11-02 14:21:22 +05:30
Stefan Kraxberger 9685ffccba Updated storing the chain so that also the reponse body is stored 2023-09-20 11:51:57 +02:00
Mzack9999 4432f63634 fixing logic 2023-09-11 18:00:14 +02:00
Mzack9999 0f6edb50be refactor hsts implementation 2023-09-11 17:53:26 +02:00
Mzack9999 8b5dc7bc0f merge conflict 2023-09-11 16:58:31 +02:00
Ramana Reddy 76adadfeb4 Add flag to preview first N characters (#1316)
* Add flag to preview first N characters

* use dynamic var for body-preview flag

* strip html tags from response

* add stripe option to remove html or xml tags

* typo

* resolve merge conflicts

* add test and update readme

* update readme

* typo

* typo no 2

* small refactor

* include body preview in jsonl output

* minor changes

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2023-09-07 15:26:54 +05:30
JoMar 179678581d Exclude WAF (#1341)
* Exclude WAF

* Add missing 's'

* Revert "Add missing 's'"

This reverts commit 467ceca320.
2023-09-05 19:53:06 +05:30
Stefan Kraxberger 964cc3bbd0 Implemented a redirect strategy which respects HSTS headers and modifies the URL accordingly if necessary 2023-08-31 10:58:51 +02:00
Stefan Kraxberger 0b702c5713 Revert "Added redirect policy which takes HSTS into account, so that even if the location would go from HTTPs to HTTP it would follow to HTTPs if an HTSTS header is present"
This reverts commit c4810ffbb7.
2023-08-31 10:44:09 +02:00
Stefan Kraxberger c4810ffbb7 Added redirect policy which takes HSTS into account, so that even if the location would go from HTTPs to HTTP it would follow to HTTPs if an HTSTS header is present 2023-08-31 10:41:20 +02:00
Stefan Kraxberger 13f78bc302 Modified storing of response chains, because it was not correct 2023-08-30 16:06:45 +02:00
Doğan Can Bakır 2718a9aab7 use mlutils 2023-07-17 06:54:00 +00:00
mzack e991ff1908 commenting out train logic 2023-06-28 11:08:09 +02:00
mzack 4ca3089d5b Merge branch 'dev' into pr/1245 2023-06-28 10:38:14 +02:00
Ramana Reddy 5f8e5d0dc5 fix tls-grab option with ztls mode (#1259)
* fix tls-grab option with ztls mode

* using canonical concatenation

* mod tidy

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
2023-06-27 20:37:47 +05:30
mzack ae1d9511df memory optimization 2023-06-22 07:27:16 +02:00
mzack 4c8ac6703e Merge branch 'dev' into pr/1245 2023-06-22 06:11:46 +02:00
Doğan Can Bakır 900f5623e8 add error page classifier 2023-06-20 14:13:48 +03:00
Mzack9999 a14af24496 Skip body read with status code 304 (#1231)
* skip body read with status code 304

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-20 14:01:51 +05:30
Mzack9999 8ea85e901d Adding prototype of tls impersonate (#1199)
* Adding prototype of tls impersonate

* updating docs

* misc update

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-06-17 16:34:19 +05:30
dependabot[bot] 7d41db16f9 chore(deps): bump github.com/projectdiscovery/cdncheck from 0.0.4-0.20220413175814-b47bc2d578b1 to 1.0.0 (#1118)
* chore(deps): bump github.com/projectdiscovery/cdncheck

Bumps [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) from 0.0.4-0.20220413175814-b47bc2d578b1 to 1.0.0.
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Changelog](https://github.com/projectdiscovery/cdncheck/blob/main/.goreleaser.yaml)
- [Commits](https://github.com/projectdiscovery/cdncheck/commits/v1.0.0)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* cdncheck update fixes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ramana Reddy <ramanaredy.manda@gmail.com>
2023-04-27 12:46:28 +05:30
Mzack9999 00d0977963 Adding Screenshot support (#1097)
* Adding Screenshot support

* adding headless body

* moving example into compilable file

* updating docs

* enabling store response with screenshot

* fixing output dir path

* fixing mod

* using native leakless

* splitting response+screenshot folders

* better handling json output

* readme update

* rel => abs path

* Added chromium into docker

* “Wipe them out. All of them.” – Darth Sidious

* utils bump

* go mod tidy

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2023-04-27 01:07:06 +05:30
Mzack9999 1defd0a9b7 Add support to skip decode (#1055)
add support to skip decode
2023-03-29 19:06:47 +05:30
Tarun Koyalwar c7456929b4 fix integration tests and upgrade deps 2023-02-08 19:50:03 +05:30
Tarun Koyalwar 3a93d2d1e8 bump url utils to v0.0.7 2023-02-07 20:15:23 +05:30
Mzack9999 db1bc2ebba Reworking favicon hash calculation (#906)
* Reworking favicon hash calculation

* removing debug code

* bumping go to 1.19

* removing automatic follow redirect

* more edge cases

* added favicon path info
2023-01-06 18:28:08 +05:30
xm1k3 88c49b812b added ztls flag (#896)
* added ztls to httpx

* adding missing dialtls

* updated readme.md

* add ztls functional test

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-12-28 23:40:22 +05:30
Tarun Koyalwar 20a9a930c2 adds base64 encoding support (#813) (#885)
* adds base64 encoding support (#813)

* rename b64 to base64
2022-11-24 21:50:37 +05:30
Mzack9999 f327e1cc77 adding generic CL test cases 2022-11-09 17:02:18 +01:00
Mzack9999 9d4f6afa8a Fixing content length bytes count 2022-11-09 16:32:56 +01:00
vrenzolaverace 686a2cced8 Update helpers libraries #850 (#851)
Use utils helpers libraries (#850)
2022-11-07 01:52:47 +05:30
Mzack9999 8af2f59d96 Fixing https with burp (#753)
* Fixing https with burp

* solving by making error non-fatal

* adding missing error comparison
2022-10-27 11:04:22 +02:00
mzack 25122cee34 fixing func signature 2022-10-25 03:22:17 +02:00
Sami 12fee4f3ee include dsl tag for all json keys (#812)
* include dsl tag for all json keys

* removed dsl tag and built list from json tag

* asn interface changed

* refactoring expression engine

- Using mapstructure for uniform mapping
- Flat key:value in expressions/variables list

* adding fields to ignore

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-10-20 13:18:01 +05:30
Sami c90b897eba list-dsl flag to list json output keys which can be further used with matchers / filters (#768)
* flag list-dsl

* variable name update

* used reflect to dnymically access DSL variables

* aligned with the changes in dev branch

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2022-09-19 22:50:40 +05:30
Ice3man 674b792770 Fixed connection pooling with cancelable timeout (#763) 2022-09-15 22:57:26 +05:30
Ice3man 9b4a2ecb0f Added tlsx integration for probing and grabbing (#767)
* Added tlsx integration for probing and grabbing

* Added additional TLS required fields
2022-09-15 20:33:51 +05:30
Sami d491201eff Issue 735+issue 739 (#757)
* adding strings.ToLower to headers

* added validation logic for flags

* validated silent flag with incompatible flags

* silent flag bug fix

* added test case

* updated test case to include multiple headers
2022-09-13 16:46:38 +05:30
forgedhallpass c48909f64f Minor fixes and improvements (#726)
* Added .idea to .gitignore

* Updated the README

* Added more verbose information on preset extractors

* removing deprecated ioutil

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2022-08-17 03:16:06 +04:00
Mzack9999 21ca2621bd Fixing and improving custom ports handling (#714)
* Fixing and improving custom ports

* adding test case from https://github.com/projectdiscovery/httpx/pull/713
2022-07-31 20:54:34 +05:30
Ice3man 90a32dd65f Added connection pooling for jarm conn + fastdialer (#707)
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2022-07-25 18:40:06 +05:30