Commit Graph

13 Commits

Author SHA1 Message Date
rustfix f4bcefb95e chore: fix some typos in comments (#1221)
Signed-off-by: rustfix <771054535@qq.com>
2024-04-16 20:41:05 +05:30
Dogan Can Bakir 3a52a8ec41 remove time.sleep from sources (#1175)
* remove` time.sleep` from sitedossier

* remove time.sleep from netlas

* remove time.sleep from gitlab

* remove time.sleep from github
2024-03-15 21:56:59 +05:30
Ramana Reddy ff7a5465d2 Add update check and self update option (#780)
* Add update flags (#779)

* remove self-update option

* - Add update check and update options (#779)
- use utils repo helper function to update binary

* fix typos

* remove pdtmutils deps

* position version log and remove quotes in logs

* replacle '%s' with %s to remove quoted string in logs
2023-03-15 17:03:36 +05:30
Owen Rumney 8f499e945b feat: surface the duration and error metrics for each source when enumerating domains (#727)
* feat: Surface the time taken in the scrape result

- Add a duration field to the `subscraping.Result` struct
- Populate the `TimeTaken` field before passing to the result channel

Resolves https://github.com/projectdiscovery/subfinder/issues/726

* fix: move the timing into the sources

- sources responsible for their own time keeping
- step over commoncrawl in the no auth test, it is consistently failing as a timeout

* feat: Add statistics

- add stats flag to the debug group
- print the time taken, number of results and errors for sources that run
- print stats for each run of the agent

* chore: fix linter error and use gologger rather than fmt print
2022-12-15 17:02:04 +05:30
forgedhallpass d2ba13d064 code review fix: Moving the apiKeys inside the Source struct 2022-08-25 13:19:44 +03:00
forgedhallpass 047136da00 Added debug messages to make troubleshooting easier
* Warn about missing keys
* Displaying the response body of failed request
2022-08-19 13:15:18 +03:00
forgedhallpass 340861831c Significant refactor around sources (part 2)
Move provider keys into sources
2022-08-10 18:21:37 +03:00
forgedhallpass 81040497c3 Significant refactor around sources (part 1)
Making the sources self-contained: information related to whether a source is default, recursive or if it requires API keys
2022-08-09 19:59:53 +03:00
Louis Dion-Marcil f5e7e620cd Improved Github subdomain matching token 2021-02-21 22:04:25 -05:00
mzack 9e7c095f42 updating gologger version 2021-02-20 19:36:47 +01:00
Mzack9999 812fc9abd5 output fix 2021-02-19 20:07:49 +01:00
Mzack9999 f6e76c0481 updating deps 2021-02-17 16:23:15 +01:00
Mzack9999 7d89b5facb go mod v2 fix 2020-09-24 19:02:02 +02:00