* Interrupt goja runtime on context cancel
* undo import formats
* do both actions in same go-routine to prevent any possible race condition
* Convert the watchdog, into a simpeler construction which still guarantees the same promises
* move the recover to the go routine, and return named return since we now can use the channel
* remove unnecessary tests
* add extra safety in case process refuses to get interrupted
* remove unnecessary ExecFuncWithTwoReturns
* remove named return values
* fix(expressions): avoid helper eval in literal checks
`hasLiteralsOnly()` currently evaluates helper
expressions while deciding whether "{{...}}"
contains unresolved variables, which makes
validation paths run side-effectful helpers.
Just replace that runtime eval with a
`Vars()` len check so unresolved-variable
detection literally stays literal (am I writing it
right?), and of course side-effect free.
Also make `Evaluate()` return template-authored
expression compile/eval errors instead of
logging and then keep continuing, so malformed
helper calls still fail in the rendering path.
Fixes#7320
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(javascript): avoid nil map panic on arg eval failure
`getArgsCopy()` logs and counts argument
evaluation failures, but continues to store Port
in `argsCopy` even when `evaluateArgs()` returns.
This causes malformed JavaScript args like
"{{base64()}}" to panic instead of returning the
original error.
Return the evaluation error immediately after
reporting it to prevent the panic and make sure
proper error propagation.
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
Restore backwards compat for JavaScript protocol
templates that omit the `Port` argument.
Regression was introduced in f4f2e9f2, which
removed the fallback for empty `Port` in
`(*Request).ExecuteWithResults`, causing templates
without `Port` to be silently skipped.
Now, if no `Port` is specified, the engine
executes the JavaScript block using the target
URL's port.
Fixes#6708.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Enhance matcher compilation with caching for regex and DSL expressions to improve performance. Update template parsing to conditionally retain raw templates based on size constraints.
* Implement caching for regex and DSL expressions in extractors and matchers to enhance performance. Introduce a buffer pool in raw requests to reduce memory allocations. Update template cache management for improved efficiency.
* feat: improve concurrency to be bound
* refactor: replace fmt.Sprintf with fmt.Fprintf for improved performance in header handling
* feat: add regex matching tests and benchmarks for performance evaluation
* feat: add prefix check in regex extraction to optimize matching process
* feat: implement regex caching mechanism to enhance performance in extractors and matchers, along with tests and benchmarks for validation
* feat: add unit tests for template execution in the core engine, enhancing test coverage and reliability
* feat: enhance error handling in template execution and improve regex caching logic for better performance
* Implement caching for regex and DSL expressions in the cache package, replacing previous sync.Map usage. Add unit tests for cache functionality, including eviction by capacity and retrieval of cached items. Update extractors and matchers to utilize the new cache system for improved performance and memory efficiency.
* Add tests for SetCapacities in cache package to ensure cache behavior on capacity changes
- Implemented TestSetCapacities_NoRebuildOnZero to verify that setting capacities to zero does not clear existing caches.
- Added TestSetCapacities_BeforeFirstUse to confirm that initial cache settings are respected and not overridden by subsequent capacity changes.
* Refactor matchers and update load test generator to use io package
- Removed maxRegexScanBytes constant from match.go.
- Replaced ioutil with io package in load_test.go for NopCloser usage.
- Restored TestValidate_AllowsInlineMultiline in load_test.go to ensure inline validation functionality.
* Add cancellation support in template execution and enhance test coverage
- Updated executeTemplateWithTargets to respect context cancellation.
- Introduced fakeTargetProvider and slowExecuter for testing.
- Added Test_executeTemplateWithTargets_RespectsCancellation to validate cancellation behavior during template execution.
* fix: remove undefined errorutil.ShowStackTrace
* feat: add make lint support and integrate with test
* refactor: migrate errorutil to errkit across codebase
- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces
* fix: improve DNS test reliability using Google DNS
- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged
* fixing logic
* removing unwanted branches in makefile
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* feat: added support for context cancellation to engine
* misc
* feat: added contexts everywhere
* misc
* misc
* use granular http timeouts and increase http timeout to 30s using multiplier
* track response header timeout in mhe
* update responseHeaderTimeout to 5sec
* skip failing windows test
---------
Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
* default threads + add threads support in dns payloads
* add threads support in network protocol
* add optional callback to override threadSetter
* fix broken fuzz integration tests
* feat http response memory optimization + reuse buffers
* update nuclei version
* feat: reuse js vm's and compile to programs
* fix failing http integration test
* remove dead code + add -jsc
* feat reuse js vms in pool with concurrency
* update comments as per review
* bug fix+ update interactsh test to look for dns interaction
* try enabling all interactsh integration tests
---------
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
* fix mem leak
* bump version tag
* http: add global resp body read limit of 4MB
* skip creating templateCtx in normal templates
* fix mem leak via retryablehttp , fastdialer
* go mod tidy
* remove unused var
* dep update
---------
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
* js protocol timeout using -timeout flag
* fix zgrab smb hang
* fix lint error
* custom timeout field in js protocol
* minor update: bound checking
* add 6 * -timeout in code protocol by default
* introduce `template-encoded` field
* remove IsCustomTemplate func
* refactor and move encoding to `MakeResultEventItem` func
* encode template in case of no results were found
* commit to last commit
* don't encode templates when`-ms` is used
* fix net read
* only return N bytes if extra available
* use ConnReadN from readerutil
* add integration test
* print unsigned warning in stderr
* fix js protocol in flow #4318
* fix integration test: url encoding issue
* fix network protocol issue + integration tests
* multiple improvements to integration test
* replace all conn.Read() from tests
* disable network-basic.yaml in windows
* disable code protocol in win CI
* fix bitwise login ps1-snippet.yaml
* hide previous matcher events in flow
* remove dead code+ update integration tests
---------
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Mikhail Epifanov