* feat(filepath): add IsPathWithinAnyDirectory func
Signed-off-by: Dwi Siswanto <git@dw1.io>
* refactor: use path-aware filesystem containment checks
Several filesystem trust boundaries relied on
lexical prefix checks to decide whether a path
fell under an allowed directory. That let sibling
paths be treated as if they were children of
trusted directories.
Replace those checks with canonical path
containment checks for helper payload loading,
template archive extraction, custom template
metadata, template path classification, and
headless screenshot output validation.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* adding more tests
* fixing tests
* adding test
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Bumps clistats, fastdialer, retryablehttp-go, dsl, networkpolicy,
ratelimit, sarif, utils, wappalyzergo, cdncheck. dsl v0.8.18 moves to
the projectdiscovery/govaluate fork, and utils v0.11.0 pulls aurora v4,
so the corresponding imports and type references are migrated.
Refs #7380.
`FindExpressions` checks marker contents with
govaluate before doing normal placeholder
replacement. That breaks valid hyphenated names
like `request-id`: govaluate reads them as
subtraction lol, and the lookup fails with a
missing request parameter.
If a marker exactly matches a key in the
replacement map, treat it as a plain placeholder
and let normal replacement handle it. Authored
expressions should still be evaluated, and
malformed helper calls should still return errors.
Fixes#7395
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(expressions): avoid helper eval in literal checks
`hasLiteralsOnly()` currently evaluates helper
expressions while deciding whether "{{...}}"
contains unresolved variables, which makes
validation paths run side-effectful helpers.
Just replace that runtime eval with a
`Vars()` len check so unresolved-variable
detection literally stays literal (am I writing it
right?), and of course side-effect free.
Also make `Evaluate()` return template-authored
expression compile/eval errors instead of
logging and then keep continuing, so malformed
helper calls still fail in the rendering path.
Fixes#7320
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(javascript): avoid nil map panic on arg eval failure
`getArgsCopy()` logs and counts argument
evaluation failures, but continues to store Port
in `argsCopy` even when `evaluateArgs()` returns.
This causes malformed JavaScript args like
"{{base64()}}" to panic instead of returning the
original error.
Return the evaluation error immediately after
reporting it to prevent the panic and make sure
proper error propagation.
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(js): respect `allow-local-file-access` in `require`
The goja `require() `function used the default
host filesystem loader which let JavaScript
templates import any local files even when
`allow-local-file-access` was disabled.
Pooled runtimes kept `require()` state around so
a module loaded during a privileged execution
could remain cached for a later restricted one.
Rebuild the require registry per execution after
setting the execution context, and route file-
backed module loads to preserve native modules
while enforcing the same sandbox rules (as
`nuclei/fs`).
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix: cross-platform sandbox path checks
Replace lexical prefix checks in the template file
sandbox with a shared path containment helper that
canonicalizes both paths before comparing them to
prevent false rejections when the configured
templates directory and the resolved file path
differ only due to symlink expansion on macOS or
path normalization on Windows.
Apply the helper in `protocolstate.NormalizePath()`
and `Options.GetValidAbsPath()` so JS `require()`-
based module loads and helper file resolution use
the same rules.
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
`interactsh.(*Client).processInteractionForRequest`
was updating the wrapped event under lock but
releasing it before calling `Operators.Execute()`,
allowing async interactsh updates to race with
matcher expr evaluation on the same `InternalEvent`
map, causing fatal concurrent map iteration/write
crashes.
Hold the event lock while populating interactsh
fields and running operators, and use a locked
`eventHash()` helper for matched-template cache
lookups that access the same event data.
Fixes#7319
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(expressions): only eval template-authored expressions
`expressions.Evaluate()` currently replaces
placeholders first and then scans the substituted
output for expressions which allows
response-derived values, including extractor
output, to be reinterpreted as DSL/helper syntax
on a second pass, which then becomes more serious
when env-backed variables are enabled.
So making this by collecting expressions from the
original template text before placeholder
substitution and then evaluating only those
original expressions after normal replacement to
keep trusted template-authored helpers working,
including placeholders nested inside helper
arguments while treating substituted response data
as literal text.
Fixes#7210
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: add resp data literal reuse test
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
When a variable value contains unresolved {{...}} markers (e.g. {{contact_id}})
and is passed through an encoding function like base64(), the markers get encoded
into an opaque blob that ContainsUnresolvedVariables cannot detect, causing
requests to fire with garbage data instead of being blocked.
Before evaluating an expression, check if its variable values contain unresolved
markers. If so, propagate those markers into the output so the existing downstream
check catches them with the correct variable names.
* fix(http): interactsh matching with `payloads`
in parallel execution.
Templates using `payloads` with Interactsh
matchers failed to detect OAST interactions
because the parallel HTTP execution path (used
when `payloads` are present) did not register
Interactsh request events, unlike the seq path.
This caused incoming interactions to lack
associated request context, preventing matchers
from running and resulting in missed detections.
Fix#5485 by wiring
`(*interactsh.Client).RequestEvent` registration
into the parallel worker goroutine, make sure both
execution paths handle Interactsh correlation
equally.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: add interactsh with `payloads` integration
Signed-off-by: Dwi Siswanto <git@dw1.io>
* test: disable interactsh-with-payloads
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
The `connectWithDSN` func used `db.Exec()` which
implicitly uses `context.Background()`[1]. This
caused the registered "nucleitcp" dialer
callback to receive a ctx missing the
`executionId`, leading to a panic during type
assertion.
Refactor `connectWithDSN` to accept `executionId`
explicitly and use it to create a `context` for
`db.PingContext()` (yeah, instead of `db.Exec()`).
And, add a defensive check in the dialer callback
to handle nil values gracefully.
Fixes#6733 regression introduced in #6296.
[1]: "Exec uses `context.Background` internally" -
https://pkg.go.dev/database/sql#DB.Exec.
Signed-off-by: Dwi Siswanto <git@dw1.io>
`MergeMaps` accounts for 11.41% of allocs (13.8
GB) in clusterbomb mode. With 1,305 combinations
per target, this function is called millions of
times in the hot path.
RCA:
* Request generator calls `MergeMaps` with single
arg on every payload combination, incurring
variadic overhead.
* Build request merges same maps multiple times
per request.
* `BuildPayloadFromOptions` recomputes static CLI
options on every call.
* Variables calls `MergeMaps` $$2×N$$ times per
variable evaluation (once in loop, once in
`evaluateVariableValue`)
Changes:
Core optimizations in maps.go:
* Pre-size merged map to avoid rehashing (30-40%
reduction)
* Add `CopyMap` for efficient single-map copy
without variadic overhead.
* Add `MergeMapsInto` for in-place mutation when
caller owns destination.
Hot path fixes:
* Replace `MergeMaps(r.currentPayloads)` with
`CopyMap(r.currentPayloads)` to eliminates
allocation on every combination iteration.
* Pre-allocate combined map once, extend in-place
during `ForEach` loop instead of creating new
map per variable (eliminates $$2×N$$ allocations
per request).
Caching with concurrency safety:
* Cache `BuildPayloadFromOptions` computation in
`sync.Map` keyed by `types.Options` ptr, but
return copy to prevent concurrent modification.
* Cost: shallow copy of ~10-20 entries vs. full
merge of vars + env (85-90% savings in typical
case)
* Clear cache in `closeInternal()` to prevent
memory leaks when SDK instances are created or
destroyed.
Estimated impact: 40-60% reduction in `MergeMaps`
allocations (5.5-8.3 GB savings from original
13.8 GB). Safe for concurrent execution and SDK
usage with multiple instances.
Signed-off-by: Dwi Siswanto <git@dw1.io>
The "Skipped X from target list as found
unresponsive permanently" message was logged on
every `(*Cache).Check()` call for hosts with
permanent errors, resulting in thousands of
duplicate log entries in verbose mode.
Wrap the log statement in `sync.Once` to match the
behavior already used for non-permanent error
logging.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(interactsh): skip DNS lookups on interactsh domains
to prevent false positives.
Prevents nuclei from resolving interactsh domains
injected in Host headers, which would cause
self-interactions to be incorrectly reported as
matches.
Changes:
* Add `GetHostname()` method to `interactsh.Client`
to expose active server domain.
* Skip CNAME DNS lookups in
`(*http.Request).addCNameIfAvailable` when
hostname matches the
`(*interactsh.Client).GetHostname`.
Fixes#6613
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(http): prevent false `interactshDomain` matches
Signed-off-by: Dwi Siswanto <git@dw1.io>
---------
Signed-off-by: Dwi Siswanto <git@dw1.io>
* Enhance matcher compilation with caching for regex and DSL expressions to improve performance. Update template parsing to conditionally retain raw templates based on size constraints.
* Implement caching for regex and DSL expressions in extractors and matchers to enhance performance. Introduce a buffer pool in raw requests to reduce memory allocations. Update template cache management for improved efficiency.
* feat: improve concurrency to be bound
* refactor: replace fmt.Sprintf with fmt.Fprintf for improved performance in header handling
* feat: add regex matching tests and benchmarks for performance evaluation
* feat: add prefix check in regex extraction to optimize matching process
* feat: implement regex caching mechanism to enhance performance in extractors and matchers, along with tests and benchmarks for validation
* feat: add unit tests for template execution in the core engine, enhancing test coverage and reliability
* feat: enhance error handling in template execution and improve regex caching logic for better performance
* Implement caching for regex and DSL expressions in the cache package, replacing previous sync.Map usage. Add unit tests for cache functionality, including eviction by capacity and retrieval of cached items. Update extractors and matchers to utilize the new cache system for improved performance and memory efficiency.
* Add tests for SetCapacities in cache package to ensure cache behavior on capacity changes
- Implemented TestSetCapacities_NoRebuildOnZero to verify that setting capacities to zero does not clear existing caches.
- Added TestSetCapacities_BeforeFirstUse to confirm that initial cache settings are respected and not overridden by subsequent capacity changes.
* Refactor matchers and update load test generator to use io package
- Removed maxRegexScanBytes constant from match.go.
- Replaced ioutil with io package in load_test.go for NopCloser usage.
- Restored TestValidate_AllowsInlineMultiline in load_test.go to ensure inline validation functionality.
* Add cancellation support in template execution and enhance test coverage
- Updated executeTemplateWithTargets to respect context cancellation.
- Introduced fakeTargetProvider and slowExecuter for testing.
- Added Test_executeTemplateWithTargets_RespectsCancellation to validate cancellation behavior during template execution.
* fix: remove undefined errorutil.ShowStackTrace
* feat: add make lint support and integrate with test
* refactor: migrate errorutil to errkit across codebase
- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces
* fix: improve DNS test reliability using Google DNS
- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged
* fixing logic
* removing unwanted branches in makefile
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Dwi Siswanto