Bumps clistats, fastdialer, retryablehttp-go, dsl, networkpolicy,
ratelimit, sarif, utils, wappalyzergo, cdncheck. dsl v0.8.18 moves to
the projectdiscovery/govaluate fork, and utils v0.11.0 pulls aurora v4,
so the corresponding imports and type references are migrated.
Refs #7380.
WithConfigFile/Bytes now decode YAML directly into a typed RuntimeConfig
struct and apply it via MergeOptions(opts). The goflags-backed reflection
overlay had too much surface area (266-line flag inventory duplicate plus
a flag-default vs DefaultOptions diff) for the actual SDK use case, which
is accepting a known set of scan knobs from cloud-shipped YAML.
RuntimeConfig schema covers tags/severity filtering, headers/vars,
interactsh, socks5-proxy, plus rate-limit/bulk-size/concurrency/timeout/
retries/rate-limit-host. Scalar knobs use *int so omitted keys preserve
the engine's existing value.
Drops internal/runner/flags.go (266 lines) and lib/config_load.go (86
lines). Reverts internal/runner/runner.go to dev (no SetupPDCPUpload
export; SDK inlines its own PDCP wrap). Keeps the LoadReportingOptionsFromBytes
extract in internal/runner/options.go for WithReportingConfigFile/Bytes.
The BindOptionFlags refactor of readConfig() caused duplicate group
headers in `nuclei -h` (CLI-only flags appended new same-named groups
after the shared ones). Suppressing the disabled-message on the CLI
was also a behavior change.
Both are scope creep. This PR is for SDK additions, not CLI refactor.
- Revert cmd/nuclei/main.go to dev. `nuclei -h` byte-identical.
- Restore "Scan results upload to cloud is disabled." in SetupPDCPUpload.
- flags.go stays for SDK-only use via lib/config_load.go.
SetupPDCPUpload returned a non-empty status string in the
upload-disabled branch, which displayExecutionInfo then logs as a
Warning on every nuclei run that doesn't request -dashboard. That
also suppresses the friendlier dashboard hint Info line.
Return empty for the default-disabled case so the Info hint fires.
The latent issue predates this PR; CodeRabbit surfaced it on the
refactor.
Extracted into flags.go from cmd/nuclei/main.go but the trailing
//nolint:all was dropped — golangci-lint trips on SA1019 in the
SDK-shared path. Restoring matches the dev branch line.
- L1: sdk_private.go references SetupPDCPUpload (was lowercase).
- L2: config_test.go test comment describes how tmpEngine is constructed
rather than incorrectly claiming "inherits from parent".
- Trim godocs and inline comments across the PR — drop restated WHAT,
keep WHY and limitations.
- Inline (*Runner).setupPDCPUpload — wrapper just delegated to the
package-level SetupPDCPUpload. Call site inlines the three lines.
- Dedupe reporting-config file load: WithReportingConfigFile and
loadImplicitReportingConfig share loadReportingConfigFromPath.
- Move config-overlay/load helpers (newConfigFlagSet, overlayConfigFromFile,
applyOverlay, loadReportingConfigFromPath, loadImplicitReportingConfig)
out of lib/config.go into lib/config_load.go.
- Drop threadSafePerScan engine mode and isThreadSafe helper. The four
new With* options no longer gate per-scan use. pd-agent doesn't enter
thread-safe mode; reintroducing the gate ships as a follow-up PR with
its own scope.
SDK consumers had to rebuild types.Options by hand or shell out to the
CLI to get config-file parity. This shares BindOptionFlags,
LoadReportingOptionsFromBytes, ApplyExporterOptionsFromTypes, and
SetupPDCPUpload between cmd/nuclei and lib/ so both paths produce
identical engine state.
WithConfigFile/Bytes apply only YAML-set fields (reflection diff against
goflags defaults) so prior With* and DefaultOptions() values survive
YAML that omits them. Per-scan ExecuteNucleiWithOpts rejects these
options via a new threadSafePerScan mode, matching the pattern for
options that only make sense at construction.
propagation, sibling isolation, and same-step
refresh, plus an integration workflow test
covering the race path.
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix(runner): write unsigned-templates WRN to stderr (#7314)
`r.Logger.Print()` is `gologger`'s LevelSilent helper — it bypasses
level filtering but routes to stdout. Using it for the unsigned-
templates "Use with caution" warning means the line collides with the
JSON / JSONL stream when the user runs with `-j`/`-jsonl`, breaking
downstream `jq` and any other consumer that expects clean JSON on
stdout.
Write the warning to os.Stderr directly (keeping the colorized [WRN]
prefix and the always-emitted semantics) so the JSON output stays
parseable. The integration suite's filterUnsignedTemplatesWarnings
helper already strips this message defensively, so no test churn is
needed.
Closes#7314.
* address review: route remaining Print() WRN sites to stderr too
CodeRabbit flagged that the rate-limit-deprecated, concurrency-vs-max-host-error,
and DAST-no-templates warnings have the same stdout-pollution problem the
unsigned-templates one had. Same fix — fmt.Fprintf to os.Stderr — applied to all
three so the stdout stream stays clean for -j / -jsonl consumers regardless of
which warning fires.
* address review: use Logger.Warning() per @dwisiswant0
Per the review on #7371, swap the four `fmt.Fprintf(os.Stderr, ...)`
WRN sites for `r.Logger.Warning().Msgf(...)`. Same stderr destination,
no duplicated WRN-prefix logic, follows the existing pattern used
across the runner package.
---------
Co-authored-by: Chris (ChrisJr404) <11917633+ChrisJr404@users.noreply.github.com>
This commit fixes a race condition where templates start executing before
the secret file's login flow completes when using -secret-file flag.
Changes:
- pkg/authprovider/authx/dynamic.go: Replace atomic.Bool-based synchronization
with sync.Once to ensure fetch callback runs exactly once and all concurrent
callers block until completion. This prevents the race where multiple goroutines
would see fetching=true and return early with nil error before secrets were ready.
- internal/runner/runner.go: Always prefetch secrets when AuthProvider exists,
removing the PreFetchSecrets flag check. This ensures authentication completes
before scanning starts regardless of flag settings.
- lib/sdk_private.go: Same change as runner.go for SDK mode consistency.
- pkg/authprovider/file_test.go: Add test for concurrent dynamic secret access
to verify fetch happens exactly once and all workers get authenticated.
- pkg/authprovider/authx/dynamic_test.go: Add concurrent fetch tests to verify
proper synchronization behavior.
Fixes#6592
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Uses metadata caching via catalog index to reduce
runtime overhead.
Add `(*Store).LoadTemplateTags` for efficient tag
aggregation.
Signed-off-by: Dwi Siswanto <git@dw1.io>
Tag listing (`-tgl`) output was moved to debug log
level, making it invisible during normal runs.
Restore tag output to default log level (Print)
for better usability.
Fixes#7142
Signed-off-by: Dwi Siswanto <git@dw1.io>
* fix: remove undefined errorutil.ShowStackTrace
* feat: add make lint support and integrate with test
* refactor: migrate errorutil to errkit across codebase
- Replace deprecated errorutil with modern errkit
- Convert error declarations from var to func for better compatibility
- Fix all SA1019 deprecation warnings
- Maintain error chain support and stack traces
* fix: improve DNS test reliability using Google DNS
- Configure test to use Google DNS (8.8.8.8) for stability
- Fix nil pointer issue in DNS client initialization
- Keep production defaults unchanged
* fixing logic
* removing unwanted branches in makefile
---------
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
* feat: fixed max-host-error blocking wrong port for template with error
* feat: log total results with time taken at end of execution
* bugfix: skip non-executed requests with progress in flow protocol
* feat: fixed request calculation in http protocol for progress
* misc adjustments
---------
Co-authored-by: Ice3man <nizamulrana@gmail.com>
Dogan Can Bakir