Compare commits

...

21 Commits

Author SHA1 Message Date
João "Pisco" Fernandes 8e7955ae89 Release 2025.7.0 2025-07-03 17:14:44 +01:00
João "Pisco" Fernandes ae197908be TUN-9540: Use numeric user id for Dockerfiles
## Summary
This commit changes the USER instruction in our Dockerfiles from using
the string "nonroot" to its numeric ID "65532".

This change is necessary because Kubernetes does not support string-based
user IDs in security contexts, requiring numeric IDs instead. The nonroot
user maps to 65532 in distroless images.
2025-07-03 11:31:04 +01:00
Devin Carr 6ec699509d TUN-9511: Add metrics for virtual DNS origin
Closes TUN-9511
2025-07-01 13:26:34 -07:00
João "Pisco" Fernandes 242fccefa4 TUN-9531: Bump go-boring from 1.24.2 to 1.24.4 2025-07-01 12:25:34 +01:00
Luis Neto d0a6318334 TUN-9161: Remove P256Kyber768Draft00PQKex curve from nonFips curve preferences
Remove P256Kyber768Draft00PQKex curve from nonFips curve preferences and add tests to verify that the advertised curves are the same as the curve preferences we set.

Closes TUN-9161
2025-07-01 12:24:59 +01:00
Devin Carr 398da8860f TUN-9473: Add --dns-resolver-addrs flag
To help support users with environments that don't work well with the
DNS local resolver's automatic resolution process for local resolver
addresses, we introduce a flag to provide them statically to the
runtime. When providing the resolver addresses, cloudflared will no
longer lookup the DNS resolver addresses and use the user input
directly.

When provided with a list of DNS resolvers larger than one, the resolver
service will randomly select one at random for each incoming request.

Closes TUN-9473
2025-06-30 15:20:32 -07:00
Devin Carr 70ed7ffc5f TUN-9470: Add OriginDialerService to include TCP
Adds an OriginDialerService that takes over the roles of both DialUDP and DialTCP 
towards the origin. This provides the possibility to leverage dialer "middleware"
to inject virtual origins, such as the DNS resolver service.

DNS Resolver service also gains access to the DialTCP operation to service TCP
DNS requests.

Minor refactoring includes changes to remove the needs previously provided by
the warp-routing configuration. This configuration cannot be disabled by cloudflared
so many of the references have been adjusted or removed.

Closes TUN-9470
2025-06-30 13:24:16 -07:00
Devin Carr 9ca8b41cf7 TUN-9472: Add virtual DNS service
Adds a new reserved service to route UDP requests towards the local DNS
resolver.

Closes TUN-9472
2025-06-27 13:09:29 -07:00
Devin Carr b4a98b13fe TUN-9469: Centralize UDP origin proxy dialing as ingress service
Introduces a new `UDPOriginProxy` interface and `UDPOriginService`
to standardize how UDP connections are dialed to origins. Allows for
future overrides of the ingress service for specific dial destinations.

Simplifies dependency injection for UDP dialing throughout both datagram
v2 and v3 by using the same ingress service. Previous invocations called
into a DialUDP function in the ingress package that was a light
wrapper over `net.DialUDP`. Now a reference is passed into both datagram
controllers that allows more control over the DialUDP method.

Closes TUN-9469
2025-06-23 18:01:15 +00:00
João "Pisco" Fernandes 64fdc52855 Release 2025.6.1 2025-06-17 10:42:18 +01:00
Devin Carr a65da54933 TUN-9371: Add logging format as JSON
Closes TUN-9371
2025-06-16 21:25:13 +00:00
João "Pisco" Fernandes 43a3ba347b TUN-9495: Remove references to cloudflare-go
## Summary
When bumping cloudflared to use go1.24, we no longer need cloudflare-go,
since most of the PQ and FIPS compliant curves are already available in go 1.24.
Therefore, we can remove everything related with installing our go toolchain.
2025-06-16 14:51:22 +00:00
João Oliveirinha 47085ee0c9 TUN-9467: add vulncheck to cloudflared
* TUN-9467: add vulncheck to cloudflared
2025-06-16 14:50:28 +00:00
João Oliveirinha a408612f26 TUN-9467: bump coredns to solve CVE
* TUN-9467: bump coredns to solve CVE
2025-06-12 10:46:10 +00:00
João "Pisco" Fernandes f8d12c9d39 Release 2025.6.0 2025-06-11 10:51:37 +01:00
Luis Neto 96ce66bd30 TUN-9016: update go to 1.24
## Summary

Update several moving parts of cloudflared build system:

* use goboring 1.24.2 in cfsetup
* update linter and fix lint issues
* update packages namely **quic-go and net**
* install script for macos
* update docker files to use go 1.24.1
* remove usage of cloudflare-go
* pin golang linter

Closes TUN-9016
2025-06-06 09:05:49 +00:00
João "Pisco" Fernandes e144eac2af TUN-9171: Use is_default_network instead of is_default to create vnet's
## Summary
The is_default field in the request body of the POST /virtual_networks endpoint has been
deprecated and should no longer be used. Clients should use the `is_default_network` field
instead for setting the default virtual network.

Closes TUN-9171
2025-06-05 15:02:21 +00:00
Devin Carr a62d63d49d Release 2025.5.0 2025-05-15 08:49:52 -07:00
Devin Carr 3bf9217de5 TUN-9319: Add dynamic loading of features to connections via ConnectionOptionsSnapshot
Make sure to enforce snapshots of features and client information for each connection
so that the feature information can change in the background. This allows for new features
to only be applied to a connection if it completely disconnects and attempts a reconnect.

Updates the feature refresh time to 1 hour from previous cloudflared versions which
refreshed every 6 hours.

Closes TUN-9319
2025-05-14 20:11:05 +00:00
Devin Carr 02705c44b2 TUN-9322: Add metric for unsupported RPC commands for datagram v3
Additionally adds support for the connection index as a label for the
datagram v3 specific tunnel metrics.

Closes TUN-9322
2025-05-13 16:11:09 +00:00
Devin Carr ce27840573 TUN-9291: Remove dynamic reloading of features for datagram v3
During a refresh of the supported features via the DNS TXT record,
cloudflared would update the internal feature list, but would not
propagate this information to the edge during a new connection.

This meant that a situation could occur in which cloudflared would
think that the client's connection could support datagram V3, and
would setup that muxer locally, but would not propagate that information
to the edge during a register connection in the `ClientInfo` of the
`ConnectionOptions`. This meant that the edge still thought that the
client was setup to support datagram V2 and since the protocols are
not backwards compatible, the local muxer for datagram V3 would reject
the incoming RPC calls.

To address this, the feature list will be fetched only once during
client bootstrapping and will persist as-is until the client is restarted.
This helps reduce the complexity involved with different connections
having possibly different sets of features when connecting to the edge.
The features will now be tied to the client and never diverge across
connections.

Also, retires the use of `support_datagram_v3` in-favor of
`support_datagram_v3_1` to reduce the risk of reusing the feature key.
The `dv3` TXT feature key is also deprecated.

Closes TUN-9291
2025-05-07 23:21:08 +00:00
1061 changed files with 54263 additions and 37014 deletions
+43 -2
View File
@@ -1,3 +1,20 @@
variables:
# Define GOPATH within the project directory to allow GitLab CI to cache it.
# By default, Go places modules in GOMODCACHE, often outside the project.
# Explicitly setting GOMODCACHE ensures it's within the cached path.
GOPATH: "$CI_PROJECT_DIR/.go"
GOMODCACHE: "$GOPATH/pkg/mod"
GO_BIN_DIR: "$GOPATH/bin"
cache:
# Cache Go modules and the binaries.
# The 'key' ensures a unique cache per branch, or you can use a fixed key
# for a shared cache across all branches if that fits your workflow.
key: "$CI_COMMIT_REF_SLUG"
paths:
- ${GOPATH}/pkg/mod/ # For Go modules
- ${GO_BIN_DIR}/
stages: [build, release]
default:
@@ -37,6 +54,23 @@ default:
when: always
- when: never
# Template for Go setup, including caching and installation
.go_setup:
image: docker-registry.cfdata.org/stash/devtools/ci-builders/golang-1.24/master:3090-3e32590@sha256:fc81df4f8322f022d93712ee40bb1e5752fdbe9868d1e5a23fd851ad6fbecb91
before_script:
- mkdir -p ${GOPATH} ${GOMODCACHE} ${GO_BIN_DIR}
- export PATH=$PATH:${GO_BIN_DIR}
- go env -w GOMODCACHE=${GOMODCACHE} # Ensure go uses the cached module path
# Check if govulncheck is already installed and install it if not
- if [ ! -f ${GO_BIN_DIR}/govulncheck ]; then
echo "govulncheck not found in cache, installing...";
go install golang.org/x/vuln/cmd/govulncheck@latest;
else
echo "govulncheck found in cache, skipping installation.";
fi
# -----------------------------------------------
# Stage 1: Build on every PR
# -----------------------------------------------
@@ -57,14 +91,21 @@ build_cloudflared_macos: &build
- '[ "${RUNNER_ARCH}" = "intel" ] && export TARGET_ARCH=amd64'
- ARCH=$(uname -m)
- echo ARCH=$ARCH - TARGET_ARCH=$TARGET_ARCH
- ./.teamcity/mac/install-cloudflare-go.sh
- export PATH="/tmp/go/bin:$PATH"
- ./.teamcity/mac/install-go.sh
- BUILD_SCRIPT=.teamcity/mac/build.sh
- if [[ ! -x ${BUILD_SCRIPT} ]] ; then exit ; fi
- set -euo pipefail
- echo "Executing ${BUILD_SCRIPT}"
- exec ${BUILD_SCRIPT}
vulncheck:
stage: build
extends: .go_setup
rules:
- !reference [.default_rules, run_on_branch]
script:
- make vulncheck
# -----------------------------------------------
# Stage 1: Build and sign only on releases
# -----------------------------------------------
+1 -1
View File
@@ -27,7 +27,7 @@ linters:
- sloglint # Ensure consistent code style when using log/slog.
- sqlclosecheck # Checks that sql.Rows, sql.Stmt, sqlx.NamedStmt, pgx.Query are closed.
- staticcheck # It's a set of rules from staticcheck. It's not the same thing as the staticcheck binary.
- tenv # Tenv is analyzer that detects using os.Setenv instead of t.Setenv since Go1.17.
- usetesting # Reports uses of functions with replacement inside the testing package.
- testableexamples # Linter checks if examples are testable (have an expected output).
- testifylint # Checks usage of github.com/stretchr/testify.
- tparallel # Tparallel detects inappropriate usage of t.Parallel() method in your Go test codes.
-8
View File
@@ -1,8 +0,0 @@
# !/usr/bin/env bash
cd /tmp
git clone -q https://github.com/cloudflare/go
cd go/src
# https://github.com/cloudflare/go/tree/af19da5605ca11f85776ef7af3384a02a315a52b is version go1.22.5-devel-cf
git checkout -q af19da5605ca11f85776ef7af3384a02a315a52b
./make.bash
@@ -2,9 +2,9 @@ rm -rf /tmp/go
export GOCACHE=/tmp/gocache
rm -rf $GOCACHE
./.teamcity/install-cloudflare-go.sh
brew install go@1.24
export PATH="/tmp/go/bin:$PATH"
go version
which go
go env
go env
+2 -2
View File
@@ -32,7 +32,7 @@ Write-Output "Running unit tests"
# Not testing with race detector because of https://github.com/golang/go/issues/61058
# We already test it on other platforms
& go test -failfast -mod=vendor ./...
go test -failfast -v -mod=vendor ./...
if ($LASTEXITCODE -ne 0) { throw "Failed unit tests" }
Write-Output "Running component tests"
@@ -44,4 +44,4 @@ if ($LASTEXITCODE -ne 0) {
python component-tests/setup.py --type cleanup
throw "Failed component tests"
}
python component-tests/setup.py --type cleanup
python component-tests/setup.py --type cleanup
-16
View File
@@ -1,16 +0,0 @@
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
Write-Output "Downloading cloudflare go..."
Set-Location "$Env:Temp"
git clone -q https://github.com/cloudflare/go
Write-Output "Building go..."
cd go/src
# https://github.com/cloudflare/go/tree/af19da5605ca11f85776ef7af3384a02a315a52b is version go1.22.5-devel-cf
git checkout -q af19da5605ca11f85776ef7af3384a02a315a52b
& ./make.bat
Write-Output "Installed"
-20
View File
@@ -1,20 +0,0 @@
$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
$GoMsiVersion = "go1.22.5.windows-amd64.msi"
Write-Output "Downloading go installer..."
Set-Location "$Env:Temp"
(New-Object System.Net.WebClient).DownloadFile(
"https://go.dev/dl/$GoMsiVersion",
"$Env:Temp\$GoMsiVersion"
)
Write-Output "Installing go..."
Install-Package "$Env:Temp\$GoMsiVersion" -Force
# Go installer updates global $PATH
go env
Write-Output "Installed"
+7 -6
View File
@@ -1,7 +1,7 @@
# use a builder image for building cloudflare
ARG TARGET_GOOS
ARG TARGET_GOARCH
FROM golang:1.22.10 as builder
FROM golang:1.24.4 AS builder
ENV GO111MODULE=on \
CGO_ENABLED=0 \
TARGET_GOOS=${TARGET_GOOS} \
@@ -16,10 +16,8 @@ WORKDIR /go/src/github.com/cloudflare/cloudflared/
# copy our sources into the builder image
COPY . .
RUN .teamcity/install-cloudflare-go.sh
# compile cloudflared
RUN PATH="/tmp/go/bin:$PATH" make cloudflared
RUN make cloudflared
# use a distroless base image with glibc
FROM gcr.io/distroless/base-debian12:nonroot
@@ -29,8 +27,11 @@ LABEL org.opencontainers.image.source="https://github.com/cloudflare/cloudflared
# copy our compiled binary
COPY --from=builder --chown=nonroot /go/src/github.com/cloudflare/cloudflared/cloudflared /usr/local/bin/
# run as non-privileged user
USER nonroot
# run as nonroot user
# We need to use numeric user id's because Kubernetes doesn't support strings:
# https://github.com/kubernetes/kubernetes/blob/v1.33.2/pkg/kubelet/kuberuntime/security_context_others.go#L49
# The `nonroot` user maps to `65532`, from: https://github.com/GoogleContainerTools/distroless/blob/main/common/variables.bzl#L18
USER 65532:65532
# command / entrypoint of container
ENTRYPOINT ["cloudflared", "--no-autoupdate"]
+7 -6
View File
@@ -1,5 +1,5 @@
# use a builder image for building cloudflare
FROM golang:1.22.10 as builder
FROM golang:1.24.4 AS builder
ENV GO111MODULE=on \
CGO_ENABLED=0 \
# the CONTAINER_BUILD envvar is used set github.com/cloudflare/cloudflared/metrics.Runtime=virtual
@@ -11,10 +11,8 @@ WORKDIR /go/src/github.com/cloudflare/cloudflared/
# copy our sources into the builder image
COPY . .
RUN .teamcity/install-cloudflare-go.sh
# compile cloudflared
RUN GOOS=linux GOARCH=amd64 PATH="/tmp/go/bin:$PATH" make cloudflared
RUN GOOS=linux GOARCH=amd64 make cloudflared
# use a distroless base image with glibc
FROM gcr.io/distroless/base-debian12:nonroot
@@ -24,8 +22,11 @@ LABEL org.opencontainers.image.source="https://github.com/cloudflare/cloudflared
# copy our compiled binary
COPY --from=builder --chown=nonroot /go/src/github.com/cloudflare/cloudflared/cloudflared /usr/local/bin/
# run as non-privileged user
USER nonroot
# run as nonroot user
# We need to use numeric user id's because Kubernetes doesn't support strings:
# https://github.com/kubernetes/kubernetes/blob/v1.33.2/pkg/kubelet/kuberuntime/security_context_others.go#L49
# The `nonroot` user maps to `65532`, from: https://github.com/GoogleContainerTools/distroless/blob/main/common/variables.bzl#L18
USER 65532:65532
# command / entrypoint of container
ENTRYPOINT ["cloudflared", "--no-autoupdate"]
+7 -6
View File
@@ -1,5 +1,5 @@
# use a builder image for building cloudflare
FROM golang:1.22.10 as builder
FROM golang:1.24.4 AS builder
ENV GO111MODULE=on \
CGO_ENABLED=0 \
# the CONTAINER_BUILD envvar is used set github.com/cloudflare/cloudflared/metrics.Runtime=virtual
@@ -11,10 +11,8 @@ WORKDIR /go/src/github.com/cloudflare/cloudflared/
# copy our sources into the builder image
COPY . .
RUN .teamcity/install-cloudflare-go.sh
# compile cloudflared
RUN GOOS=linux GOARCH=arm64 PATH="/tmp/go/bin:$PATH" make cloudflared
RUN GOOS=linux GOARCH=arm64 make cloudflared
# use a distroless base image with glibc
FROM gcr.io/distroless/base-debian12:nonroot-arm64
@@ -24,8 +22,11 @@ LABEL org.opencontainers.image.source="https://github.com/cloudflare/cloudflared
# copy our compiled binary
COPY --from=builder --chown=nonroot /go/src/github.com/cloudflare/cloudflared/cloudflared /usr/local/bin/
# run as non-privileged user
USER nonroot
# run as nonroot user
# We need to use numeric user id's because Kubernetes doesn't support strings:
# https://github.com/kubernetes/kubernetes/blob/v1.33.2/pkg/kubelet/kuberuntime/security_context_others.go#L49
# The `nonroot` user maps to `65532`, from: https://github.com/GoogleContainerTools/distroless/blob/main/common/variables.bzl#L18
USER 65532:65532
# command / entrypoint of container
ENTRYPOINT ["cloudflared", "--no-autoupdate"]
+5 -12
View File
@@ -56,8 +56,6 @@ PACKAGE_DIR := $(CURDIR)/packaging
PREFIX := /usr
INSTALL_BINDIR := $(PREFIX)/bin/
INSTALL_MANDIR := $(PREFIX)/share/man/man1/
CF_GO_PATH := /tmp/go
PATH := $(CF_GO_PATH)/bin:$(PATH)
LOCAL_ARCH ?= $(shell uname -m)
ifneq ($(GOARCH),)
@@ -129,6 +127,10 @@ all: cloudflared test
clean:
go clean
.PHONY: vulncheck
vulncheck:
@govulncheck ./...
.PHONY: cloudflared
cloudflared:
ifeq ($(FIPS), true)
@@ -179,19 +181,10 @@ fuzz:
@go test -fuzz=FuzzNewIdentity -fuzztime=600s ./tracing
@go test -fuzz=FuzzNewAccessValidator -fuzztime=600s ./validation
.PHONY: install-go
install-go:
rm -rf ${CF_GO_PATH}
./.teamcity/install-cloudflare-go.sh
.PHONY: cleanup-go
cleanup-go:
rm -rf ${CF_GO_PATH}
cloudflared.1: cloudflared_man_template
sed -e 's/\$${VERSION}/$(VERSION)/; s/\$${DATE}/$(DATE)/' cloudflared_man_template > cloudflared.1
install: install-go cloudflared cloudflared.1 cleanup-go
install: cloudflared cloudflared.1
mkdir -p $(DESTDIR)$(INSTALL_BINDIR) $(DESTDIR)$(INSTALL_MANDIR)
install -m755 cloudflared $(DESTDIR)$(INSTALL_BINDIR)/cloudflared
install -m644 cloudflared.1 $(DESTDIR)$(INSTALL_MANDIR)/cloudflared.1
+2 -2
View File
@@ -31,7 +31,7 @@ Downloads are available as standalone binaries, a Docker image, and Debian, RPM,
* Binaries, Debian, and RPM packages for Linux [can be found here](https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/installation#linux)
* A Docker image of `cloudflared` is [available on DockerHub](https://hub.docker.com/r/cloudflare/cloudflared)
* You can install on Windows machines with the [steps here](https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/installation#windows)
* To build from source, first you need to download the go toolchain by running `./.teamcity/install-cloudflare-go.sh` and follow the output. Then you can run `make cloudflared`
* To build from source, install the required version of go, mentioned in the [Development](#development) section below. Then you can run `make cloudflared`.
User documentation for Cloudflare Tunnel can be found at https://developers.cloudflare.com/cloudflare-one/connections/connect-apps
@@ -62,7 +62,7 @@ For example, as of January 2023 Cloudflare will support cloudflared version 2023
### Requirements
- [GNU Make](https://www.gnu.org/software/make/)
- [capnp](https://capnproto.org/install.html)
- [cloudflare go toolchain](https://github.com/cloudflare/go)
- [go >= 1.24](https://go.dev/doc/install)
- Optional tools:
- [capnpc-go](https://pkg.go.dev/zombiezen.com/go/capnproto2/capnpc-go)
- [goimports](https://pkg.go.dev/golang.org/x/tools/cmd/goimports)
+25
View File
@@ -1,3 +1,28 @@
2025.7.0
- 2025-07-03 TUN-9540: Use numeric user id for Dockerfiles
- 2025-07-01 TUN-9161: Remove P256Kyber768Draft00PQKex curve from nonFips curve preferences
- 2025-07-01 TUN-9531: Bump go-boring from 1.24.2 to 1.24.4
- 2025-07-01 TUN-9511: Add metrics for virtual DNS origin
- 2025-06-30 TUN-9470: Add OriginDialerService to include TCP
- 2025-06-30 TUN-9473: Add --dns-resolver-addrs flag
- 2025-06-27 TUN-9472: Add virtual DNS service
- 2025-06-23 TUN-9469: Centralize UDP origin proxy dialing as ingress service
2025.6.1
- 2025-06-16 TUN-9467: add vulncheck to cloudflared
- 2025-06-16 TUN-9495: Remove references to cloudflare-go
- 2025-06-16 TUN-9371: Add logging format as JSON
- 2025-06-12 TUN-9467: bump coredns to solve CVE
2025.6.0
- 2025-06-06 TUN-9016: update go to 1.24
- 2025-06-05 TUN-9171: Use `is_default_network` instead of `is_default` to create vnet's
2025.5.0
- 2025-05-14 TUN-9319: Add dynamic loading of features to connections via ConnectionOptionsSnapshot
- 2025-05-13 TUN-9322: Add metric for unsupported RPC commands for datagram v3
- 2025-05-07 TUN-9291: Remove dynamic reloading of features for datagram v3
2025.4.2
- 2025-04-30 chore: Do not use gitlab merge request pipelines
- 2025-04-30 DEVTOOLS-16383: Create GitlabCI pipeline to release Mac builds
+1 -1
View File
@@ -16,7 +16,7 @@ import (
type NewVirtualNetwork struct {
Name string `json:"name"`
Comment string `json:"comment"`
IsDefault bool `json:"is_default"`
IsDefault bool `json:"is_default_network"`
}
type VirtualNetwork struct {
+2 -2
View File
@@ -1,4 +1,4 @@
pinned_go: &pinned_go go-boring=1.22.10-1
pinned_go: &pinned_go go-boring=1.24.4-1
build_dir: &build_dir /cfsetup_build
default-flavor: bookworm
@@ -13,7 +13,7 @@ bullseye: &bullseye
- rubygem-fpm
- rpm
- libffi-dev
- golangci-lint
- golangci-lint=1.64.8-2
pre-cache: &build_pre_cache
- export GOCACHE=/cfsetup_build/.cache/go-build
- go install golang.org/x/tools/cmd/goimports@v0.30.0
+74
View File
@@ -0,0 +1,74 @@
package client
import (
"fmt"
"net"
"github.com/google/uuid"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
// Config captures the local client runtime configuration.
type Config struct {
ConnectorID uuid.UUID
Version string
Arch string
featureSelector features.FeatureSelector
}
func NewConfig(version string, arch string, featureSelector features.FeatureSelector) (*Config, error) {
connectorID, err := uuid.NewRandom()
if err != nil {
return nil, fmt.Errorf("unable to generate a connector UUID: %w", err)
}
return &Config{
ConnectorID: connectorID,
Version: version,
Arch: arch,
featureSelector: featureSelector,
}, nil
}
// ConnectionOptionsSnapshot is a snapshot of the current client information used to initialize a connection.
//
// The FeatureSnapshot is the features that are available for this connection. At the client level they may
// change, but they will not change within the scope of this struct.
type ConnectionOptionsSnapshot struct {
client pogs.ClientInfo
originLocalIP net.IP
numPreviousAttempts uint8
FeatureSnapshot features.FeatureSnapshot
}
func (c *Config) ConnectionOptionsSnapshot(originIP net.IP, previousAttempts uint8) *ConnectionOptionsSnapshot {
snapshot := c.featureSelector.Snapshot()
return &ConnectionOptionsSnapshot{
client: pogs.ClientInfo{
ClientID: c.ConnectorID[:],
Version: c.Version,
Arch: c.Arch,
Features: snapshot.FeaturesList,
},
originLocalIP: originIP,
numPreviousAttempts: previousAttempts,
FeatureSnapshot: snapshot,
}
}
func (c ConnectionOptionsSnapshot) ConnectionOptions() *pogs.ConnectionOptions {
return &pogs.ConnectionOptions{
Client: c.client,
OriginLocalIP: c.originLocalIP,
ReplaceExisting: false,
CompressionQuality: 0,
NumPreviousAttempts: c.numPreviousAttempts,
}
}
func (c ConnectionOptionsSnapshot) LogFields(event *zerolog.Event) *zerolog.Event {
return event.Strs("features", c.client.Features)
}
+50
View File
@@ -0,0 +1,50 @@
package client
import (
"net"
"testing"
"github.com/stretchr/testify/require"
"github.com/cloudflare/cloudflared/features"
)
func TestGenerateConnectionOptions(t *testing.T) {
version := "1234"
arch := "linux_amd64"
originIP := net.ParseIP("192.168.1.1")
var previousAttempts uint8 = 4
config, err := NewConfig(version, arch, &mockFeatureSelector{})
require.NoError(t, err)
require.Equal(t, version, config.Version)
require.Equal(t, arch, config.Arch)
// Validate ConnectionOptionsSnapshot fields
connOptions := config.ConnectionOptionsSnapshot(originIP, previousAttempts)
require.Equal(t, version, connOptions.client.Version)
require.Equal(t, arch, connOptions.client.Arch)
require.Equal(t, config.ConnectorID[:], connOptions.client.ClientID)
// Vaidate snapshot feature fields against the connOptions generated
snapshot := config.featureSelector.Snapshot()
require.Equal(t, features.DatagramV3, snapshot.DatagramVersion)
require.Equal(t, features.DatagramV3, connOptions.FeatureSnapshot.DatagramVersion)
pogsConnOptions := connOptions.ConnectionOptions()
require.Equal(t, connOptions.client, pogsConnOptions.Client)
require.Equal(t, originIP, pogsConnOptions.OriginLocalIP)
require.False(t, pogsConnOptions.ReplaceExisting)
require.Equal(t, uint8(0), pogsConnOptions.CompressionQuality)
require.Equal(t, previousAttempts, pogsConnOptions.NumPreviousAttempts)
}
type mockFeatureSelector struct{}
func (m *mockFeatureSelector) Snapshot() features.FeatureSnapshot {
return features.FeatureSnapshot{
PostQuantum: features.PostQuantumPrefer,
DatagramVersion: features.DatagramV3,
FeaturesList: []string{features.FeaturePostQuantum, features.FeatureDatagramV3_1},
}
}
+14 -6
View File
@@ -4,25 +4,32 @@ import (
"github.com/urfave/cli/v2"
"github.com/urfave/cli/v2/altsrc"
cfdflags "github.com/cloudflare/cloudflared/cmd/cloudflared/flags"
"github.com/cloudflare/cloudflared/cmd/cloudflared/flags"
)
var (
debugLevelWarning = "At debug level cloudflared will log request URL, method, protocol, content length, as well as, all request and response headers. " +
"This can expose sensitive information in your logs."
FlagLogOutput = &cli.StringFlag{
Name: flags.LogFormatOutput,
Usage: "Output format for the logs (default, json)",
Value: flags.LogFormatOutputValueDefault,
EnvVars: []string{"TUNNEL_MANAGEMENT_OUTPUT", "TUNNEL_LOG_OUTPUT"},
}
)
func ConfigureLoggingFlags(shouldHide bool) []cli.Flag {
return []cli.Flag{
altsrc.NewStringFlag(&cli.StringFlag{
Name: cfdflags.LogLevel,
Name: flags.LogLevel,
Value: "info",
Usage: "Application logging level {debug, info, warn, error, fatal}. " + debugLevelWarning,
EnvVars: []string{"TUNNEL_LOGLEVEL"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: cfdflags.TransportLogLevel,
Name: flags.TransportLogLevel,
Aliases: []string{"proto-loglevel"}, // This flag used to be called proto-loglevel
Value: "info",
Usage: "Transport logging level(previously called protocol logging level) {debug, info, warn, error, fatal}",
@@ -30,22 +37,23 @@ func ConfigureLoggingFlags(shouldHide bool) []cli.Flag {
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: cfdflags.LogFile,
Name: flags.LogFile,
Usage: "Save application log to this file for reporting issues.",
EnvVars: []string{"TUNNEL_LOGFILE"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: cfdflags.LogDirectory,
Name: flags.LogDirectory,
Usage: "Save application log to this directory for reporting issues.",
EnvVars: []string{"TUNNEL_LOGDIRECTORY"},
Hidden: shouldHide,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: cfdflags.TraceOutput,
Name: flags.TraceOutput,
Usage: "Name of trace output file, generated when cloudflared stops.",
EnvVars: []string{"TUNNEL_TRACE_OUTPUT"},
Hidden: shouldHide,
}),
FlagLogOutput,
}
}
+8
View File
@@ -138,6 +138,11 @@ const (
// LogDirectory is the command line flag to define the directory where application logs will be stored.
LogDirectory = "log-directory"
// LogFormatOutput allows the command line logs to be output as JSON.
LogFormatOutput = "output"
LogFormatOutputValueDefault = "default"
LogFormatOutputValueJSON = "json"
// TraceOutput is the command line flag to set the name of trace output file
TraceOutput = "trace-output"
@@ -152,4 +157,7 @@ const (
// ApiURL is the command line flag used to define the base URL of the API
ApiURL = "api-url"
// Virtual DNS resolver service resolver addresses to use instead of dynamically fetching them from the OS.
VirtualDNSServiceResolverAddresses = "dns-resolver-addrs"
)
+44 -9
View File
@@ -4,6 +4,7 @@ package main
import (
"fmt"
"io"
"os"
"github.com/rs/zerolog"
@@ -15,7 +16,7 @@ import (
"github.com/cloudflare/cloudflared/logger"
)
func runApp(app *cli.App, graceShutdownC chan struct{}) {
func runApp(app *cli.App, _ chan struct{}) {
app.Commands = append(app.Commands, &cli.Command{
Name: "service",
Usage: "Manages the cloudflared system service",
@@ -35,7 +36,7 @@ func runApp(app *cli.App, graceShutdownC chan struct{}) {
},
},
})
app.Run(os.Args)
_ = app.Run(os.Args)
}
// The directory and files that are used by the service.
@@ -97,6 +98,7 @@ WantedBy=timers.target
var sysvTemplate = ServiceTemplate{
Path: "/etc/init.d/cloudflared",
FileMode: 0755,
// nolint: dupword
Content: `#!/bin/sh
# For RedHat and cousins:
# chkconfig: 2345 99 01
@@ -184,13 +186,11 @@ exit 0
`,
}
var (
noUpdateServiceFlag = &cli.BoolFlag{
Name: "no-update-service",
Usage: "Disable auto-update of the cloudflared linux service, which restarts the server to upgrade for new versions.",
Value: false,
}
)
var noUpdateServiceFlag = &cli.BoolFlag{
Name: "no-update-service",
Usage: "Disable auto-update of the cloudflared linux service, which restarts the server to upgrade for new versions.",
Value: false,
}
func isSystemd() bool {
if _, err := os.Stat("/run/systemd/system"); err == nil {
@@ -430,3 +430,38 @@ func uninstallSysv(log *zerolog.Logger) error {
}
return nil
}
func ensureConfigDirExists(configDir string) error {
ok, err := config.FileExists(configDir)
if !ok && err == nil {
err = os.Mkdir(configDir, 0755)
}
return err
}
func copyFile(src, dest string) error {
srcFile, err := os.Open(src)
if err != nil {
return err
}
defer srcFile.Close()
destFile, err := os.Create(dest)
if err != nil {
return err
}
ok := false
defer func() {
destFile.Close()
if !ok {
_ = os.Remove(dest)
}
}()
if _, err := io.Copy(destFile, srcFile); err != nil {
return err
}
ok = true
return nil
}
-37
View File
@@ -11,8 +11,6 @@ import (
"text/template"
homedir "github.com/mitchellh/go-homedir"
"github.com/cloudflare/cloudflared/config"
)
type ServiceTemplate struct {
@@ -109,38 +107,3 @@ func runCommand(command string, args ...string) error {
}
return nil
}
func ensureConfigDirExists(configDir string) error {
ok, err := config.FileExists(configDir)
if !ok && err == nil {
err = os.Mkdir(configDir, 0755)
}
return err
}
func copyFile(src, dest string) error {
srcFile, err := os.Open(src)
if err != nil {
return err
}
defer srcFile.Close()
destFile, err := os.Create(dest)
if err != nil {
return err
}
ok := false
defer func() {
destFile.Close()
if !ok {
_ = os.Remove(dest)
}
}()
if _, err := io.Copy(destFile, srcFile); err != nil {
return err
}
ok = true
return nil
}
+17 -10
View File
@@ -4,6 +4,7 @@ import (
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
@@ -97,12 +98,6 @@ func buildTailCommand(subcommands []*cli.Command) *cli.Command {
Value: "",
EnvVars: []string{"TUNNEL_MANAGEMENT_TOKEN"},
},
&cli.StringFlag{
Name: "output",
Usage: "Output format for the logs (default, json)",
Value: "default",
EnvVars: []string{"TUNNEL_MANAGEMENT_OUTPUT"},
},
&cli.StringFlag{
Name: "management-hostname",
Usage: "Management hostname to signify incoming management requests",
@@ -128,6 +123,7 @@ func buildTailCommand(subcommands []*cli.Command) *cli.Command {
EnvVars: []string{"TUNNEL_ORIGIN_CERT"},
Value: credentials.FindDefaultOriginCertPath(),
},
cliutil.FlagLogOutput,
},
Subcommands: subcommands,
}
@@ -171,10 +167,21 @@ func createLogger(c *cli.Context) *zerolog.Logger {
if levelErr != nil {
level = zerolog.InfoLevel
}
log := zerolog.New(zerolog.ConsoleWriter{
Out: colorable.NewColorable(os.Stderr),
TimeFormat: time.RFC3339,
}).With().Timestamp().Logger().Level(level)
var writer io.Writer
switch c.String(cfdflags.LogFormatOutput) {
case cfdflags.LogFormatOutputValueJSON:
// zerolog by default outputs as JSON
writer = os.Stderr
case cfdflags.LogFormatOutputValueDefault:
// "default" and unset use the same logger output format
fallthrough
default:
writer = zerolog.ConsoleWriter{
Out: colorable.NewColorable(os.Stderr),
TimeFormat: time.RFC3339,
}
}
log := zerolog.New(writer).With().Timestamp().Logger().Level(level)
return &log
}
+4 -12
View File
@@ -15,7 +15,6 @@ import (
"github.com/coreos/go-systemd/v22/daemon"
"github.com/facebookgo/grace/gracenet"
"github.com/getsentry/sentry-go"
"github.com/google/uuid"
"github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/rs/zerolog"
@@ -446,14 +445,7 @@ func StartServer(
log.Err(err).Msg("Couldn't start tunnel")
return err
}
var clientID uuid.UUID
if tunnelConfig.NamedTunnel != nil {
clientID, err = uuid.FromBytes(tunnelConfig.NamedTunnel.Client.ClientID)
if err != nil {
// set to nil for classic tunnels
clientID = uuid.Nil
}
}
connectorID := tunnelConfig.ClientConfig.ConnectorID
// Disable ICMP packet routing for quick tunnels
if quickTunnelURL != "" {
@@ -471,7 +463,7 @@ func StartServer(
c.String("management-hostname"),
c.Bool("management-diagnostics"),
serviceIP,
clientID,
connectorID,
c.String(cfdflags.ConnectorLabel),
logger.ManagementLogger.Log,
logger.ManagementLogger,
@@ -503,14 +495,14 @@ func StartServer(
sources = append(sources, ipv6.String())
}
readinessServer := metrics.NewReadyServer(clientID, tracker)
readinessServer := metrics.NewReadyServer(connectorID, tracker)
cliFlags := nonSecretCliFlags(log, c, nonSecretFlagsList)
diagnosticHandler := diagnostic.NewDiagnosticHandler(
log,
0,
diagnostic.NewSystemCollectorImpl(buildInfo.CloudflaredVersion),
tunnelConfig.NamedTunnel.Credentials.TunnelID,
clientID,
connectorID,
tracker,
cliFlags,
sources,
-15
View File
@@ -1,15 +0,0 @@
package tunnel
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/cloudflare/cloudflared/features"
)
func TestDedup(t *testing.T) {
expected := []string{"a", "b"}
actual := features.Dedup([]string{"a", "b", "a"})
require.ElementsMatch(t, expected, actual)
}
+62 -28
View File
@@ -10,13 +10,14 @@ import (
"strings"
"time"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"github.com/urfave/cli/v2/altsrc"
"golang.org/x/term"
"github.com/cloudflare/cloudflared/client"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/cmd/cloudflared/flags"
"github.com/cloudflare/cloudflared/config"
@@ -25,6 +26,7 @@ import (
"github.com/cloudflare/cloudflared/edgediscovery/allregions"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/ingress/origins"
"github.com/cloudflare/cloudflared/orchestration"
"github.com/cloudflare/cloudflared/supervisor"
"github.com/cloudflare/cloudflared/tlsconfig"
@@ -125,27 +127,29 @@ func prepareTunnelConfig(
observer *connection.Observer,
namedTunnel *connection.TunnelProperties,
) (*supervisor.TunnelConfig, *orchestration.Config, error) {
clientID, err := uuid.NewRandom()
transportProtocol := c.String(flags.Protocol)
isPostQuantumEnforced := c.Bool(flags.PostQuantum)
featureSelector, err := features.NewFeatureSelector(ctx, namedTunnel.Credentials.AccountTag, c.StringSlice(flags.Features), isPostQuantumEnforced, log)
if err != nil {
return nil, nil, errors.Wrap(err, "can't generate connector UUID")
return nil, nil, errors.Wrap(err, "Failed to create feature selector")
}
log.Info().Msgf("Generated Connector ID: %s", clientID)
clientConfig, err := client.NewConfig(info.Version(), info.OSArch(), featureSelector)
if err != nil {
return nil, nil, err
}
log.Info().Msgf("Generated Connector ID: %s", clientConfig.ConnectorID)
tags, err := NewTagSliceFromCLI(c.StringSlice(flags.Tag))
if err != nil {
log.Err(err).Msg("Tag parse failure")
return nil, nil, errors.Wrap(err, "Tag parse failure")
}
tags = append(tags, pogs.Tag{Name: "ID", Value: clientID.String()})
tags = append(tags, pogs.Tag{Name: "ID", Value: clientConfig.ConnectorID.String()})
transportProtocol := c.String(flags.Protocol)
isPostQuantumEnforced := c.Bool(flags.PostQuantum)
featureSelector, err := features.NewFeatureSelector(ctx, namedTunnel.Credentials.AccountTag, c.StringSlice("features"), c.Bool("post-quantum"), log)
if err != nil {
return nil, nil, errors.Wrap(err, "Failed to create feature selector")
}
clientFeatures := featureSelector.ClientFeatures()
pqMode := featureSelector.PostQuantumMode()
clientFeatures := featureSelector.Snapshot()
pqMode := clientFeatures.PostQuantum
if pqMode == features.PostQuantumStrict {
// Error if the user tries to force a non-quic transport protocol
if transportProtocol != connection.AutoSelectFlag && transportProtocol != connection.QUIC.String() {
@@ -154,12 +158,6 @@ func prepareTunnelConfig(
transportProtocol = connection.QUIC.String()
}
namedTunnel.Client = pogs.ClientInfo{
ClientID: clientID[:],
Features: clientFeatures,
Version: info.Version(),
Arch: info.OSArch(),
}
cfg := config.GetConfiguration()
ingressRules, err := ingress.ParseIngressFromConfigAndCLI(cfg, c, log)
if err != nil {
@@ -223,11 +221,30 @@ func prepareTunnelConfig(
resolvedRegion = endpoint
}
warpRoutingConfig := ingress.NewWarpRoutingConfig(&cfg.WarpRouting)
// Setup origin dialer service and virtual services
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: ingress.NewDialer(warpRoutingConfig),
TCPWriteTimeout: c.Duration(flags.WriteStreamTimeout),
}, log)
// Setup DNS Resolver Service
originMetrics := origins.NewMetrics(prometheus.DefaultRegisterer)
dnsResolverAddrs := c.StringSlice(flags.VirtualDNSServiceResolverAddresses)
dnsService := origins.NewDNSResolverService(origins.NewDNSDialer(), log, originMetrics)
if len(dnsResolverAddrs) > 0 {
addrs, err := parseResolverAddrPorts(dnsResolverAddrs)
if err != nil {
return nil, nil, fmt.Errorf("invalid %s provided: %w", flags.VirtualDNSServiceResolverAddresses, err)
}
dnsService = origins.NewStaticDNSResolverService(addrs, origins.NewDNSDialer(), log, originMetrics)
}
originDialerService.AddReservedService(dnsService, []netip.AddrPort{origins.VirtualDNSServiceAddr})
tunnelConfig := &supervisor.TunnelConfig{
ClientConfig: clientConfig,
GracePeriod: gracePeriod,
ReplaceExisting: c.Bool(flags.Force),
OSArch: info.OSArch(),
ClientID: clientID.String(),
EdgeAddrs: c.StringSlice(flags.Edge),
Region: resolvedRegion,
EdgeIPVersion: edgeIPVersion,
@@ -246,13 +263,14 @@ func prepareTunnelConfig(
NamedTunnel: namedTunnel,
ProtocolSelector: protocolSelector,
EdgeTLSConfigs: edgeTLSConfigs,
FeatureSelector: featureSelector,
MaxEdgeAddrRetries: uint8(c.Int(flags.MaxEdgeAddrRetries)), // nolint: gosec
RPCTimeout: c.Duration(flags.RpcTimeout),
WriteStreamTimeout: c.Duration(flags.WriteStreamTimeout),
DisableQUICPathMTUDiscovery: c.Bool(flags.QuicDisablePathMTUDiscovery),
QUICConnectionLevelFlowControlLimit: c.Uint64(flags.QuicConnLevelFlowControlLimit),
QUICStreamLevelFlowControlLimit: c.Uint64(flags.QuicStreamLevelFlowControlLimit),
OriginDNSService: dnsService,
OriginDialerService: originDialerService,
}
icmpRouter, err := newICMPRouter(c, log)
if err != nil {
@@ -261,10 +279,10 @@ func prepareTunnelConfig(
tunnelConfig.ICMPRouterServer = icmpRouter
}
orchestratorConfig := &orchestration.Config{
Ingress: &ingressRules,
WarpRouting: ingress.NewWarpRoutingConfig(&cfg.WarpRouting),
ConfigurationFlags: parseConfigFlags(c),
WriteTimeout: tunnelConfig.WriteStreamTimeout,
Ingress: &ingressRules,
WarpRouting: warpRoutingConfig,
OriginDialerService: originDialerService,
ConfigurationFlags: parseConfigFlags(c),
}
return tunnelConfig, orchestratorConfig, nil
}
@@ -501,3 +519,19 @@ func findLocalAddr(dst net.IP, port int) (netip.Addr, error) {
localAddr := localAddrPort.Addr()
return localAddr, nil
}
func parseResolverAddrPorts(input []string) ([]netip.AddrPort, error) {
// We don't allow more than 10 resolvers to be provided statically for the resolver service.
if len(input) > 10 {
return nil, errors.New("too many addresses provided, max: 10")
}
addrs := make([]netip.AddrPort, 0, len(input))
for _, val := range input {
addr, err := netip.ParseAddrPort(val)
if err != nil {
return nil, err
}
addrs = append(addrs, addr)
}
return addrs, nil
}
+7 -1
View File
@@ -241,6 +241,11 @@ var (
Usage: "Overrides the remote configuration for max active private network flows (TCP/UDP) that this cloudflared instance supports",
EnvVars: []string{"TUNNEL_MAX_ACTIVE_FLOWS"},
}
dnsResolverAddrsFlag = &cli.StringSliceFlag{
Name: flags.VirtualDNSServiceResolverAddresses,
Usage: "Overrides the dynamic DNS resolver resolution to use these address:port's instead.",
EnvVars: []string{"TUNNEL_DNS_RESOLVER_ADDRS"},
}
)
func buildCreateCommand() *cli.Command {
@@ -718,6 +723,7 @@ func buildRunCommand() *cli.Command {
icmpv4SrcFlag,
icmpv6SrcFlag,
maxActiveFlowsFlag,
dnsResolverAddrsFlag,
}
flags = append(flags, configureProxyFlags(false)...)
return &cli.Command{
@@ -761,7 +767,7 @@ func runCommand(c *cli.Context) error {
if tokenFile := c.String(TunnelTokenFileFlag); tokenFile != "" {
data, err := os.ReadFile(tokenFile)
if err != nil {
return cliutil.UsageError("Failed to read token file: " + err.Error())
return cliutil.UsageError("Failed to read token file: %s", err.Error())
}
tokenStr = strings.TrimSpace(string(data))
}
+2 -2
View File
@@ -190,7 +190,7 @@ func installWindowsService(c *cli.Context) error {
log := zeroLogger.With().Str(LogFieldWindowsServiceName, windowsServiceName).Logger()
if err == nil {
s.Close()
return fmt.Errorf(serviceAlreadyExistsWarn(windowsServiceName))
return errors.New(serviceAlreadyExistsWarn(windowsServiceName))
}
extraArgs, err := getServiceExtraArgsFromCliArgs(c, &log)
if err != nil {
@@ -238,7 +238,7 @@ func uninstallWindowsService(c *cli.Context) error {
defer m.Disconnect()
s, err := m.OpenService(windowsServiceName)
if err != nil {
return fmt.Errorf("Agent service %s is not installed, so it could not be uninstalled", windowsServiceName)
return fmt.Errorf("agent service %s is not installed, so it could not be uninstalled", windowsServiceName)
}
defer s.Close()
-1
View File
@@ -57,7 +57,6 @@ type Orchestrator interface {
type TunnelProperties struct {
Credentials Credentials
Client pogs.ClientInfo
QuickTunnelUrl string
}
+3 -3
View File
@@ -10,7 +10,7 @@ import (
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/tunnelrpc"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
// registerClient derives a named tunnel rpc client that can then be used to register and unregister connections.
@@ -36,7 +36,7 @@ type controlStream struct {
// ControlStreamHandler registers connections with origintunneld and initiates graceful shutdown.
type ControlStreamHandler interface {
// ServeControlStream handles the control plane of the transport in the current goroutine calling this
ServeControlStream(ctx context.Context, rw io.ReadWriteCloser, connOptions *tunnelpogs.ConnectionOptions, tunnelConfigGetter TunnelConfigJSONGetter) error
ServeControlStream(ctx context.Context, rw io.ReadWriteCloser, connOptions *pogs.ConnectionOptions, tunnelConfigGetter TunnelConfigJSONGetter) error
// IsStopped tells whether the method above has finished
IsStopped() bool
}
@@ -78,7 +78,7 @@ func NewControlStream(
func (c *controlStream) ServeControlStream(
ctx context.Context,
rw io.ReadWriteCloser,
connOptions *tunnelpogs.ConnectionOptions,
connOptions *pogs.ConnectionOptions,
tunnelConfigGetter TunnelConfigJSONGetter,
) error {
registrationClient := c.registerClientFunc(ctx, rw, c.registerTimeout)
+4 -4
View File
@@ -16,10 +16,10 @@ import (
"github.com/rs/zerolog"
"golang.org/x/net/http2"
"github.com/cloudflare/cloudflared/client"
cfdflow "github.com/cloudflare/cloudflared/flow"
"github.com/cloudflare/cloudflared/tracing"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
// note: these constants are exported so we can reuse them in the edge-side code
@@ -39,7 +39,7 @@ type HTTP2Connection struct {
conn net.Conn
server *http2.Server
orchestrator Orchestrator
connOptions *tunnelpogs.ConnectionOptions
connOptions *client.ConnectionOptionsSnapshot
observer *Observer
connIndex uint8
@@ -54,7 +54,7 @@ type HTTP2Connection struct {
func NewHTTP2Connection(
conn net.Conn,
orchestrator Orchestrator,
connOptions *tunnelpogs.ConnectionOptions,
connOptions *client.ConnectionOptionsSnapshot,
observer *Observer,
connIndex uint8,
controlStreamHandler ControlStreamHandler,
@@ -118,7 +118,7 @@ func (c *HTTP2Connection) ServeHTTP(w http.ResponseWriter, r *http.Request) {
var requestErr error
switch connType {
case TypeControlStream:
requestErr = c.controlStreamHandler.ServeControlStream(r.Context(), respWriter, c.connOptions, c.orchestrator)
requestErr = c.controlStreamHandler.ServeControlStream(r.Context(), respWriter, c.connOptions.ConnectionOptions(), c.orchestrator)
if requestErr != nil {
c.controlStreamErr = requestErr
}
+14 -15
View File
@@ -20,19 +20,18 @@ import (
"github.com/stretchr/testify/require"
"golang.org/x/net/http2"
"github.com/cloudflare/cloudflared/client"
"github.com/cloudflare/cloudflared/tracing"
"github.com/cloudflare/cloudflared/tunnelrpc"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
var (
testTransport = http2.Transport{}
)
var testTransport = http2.Transport{}
func newTestHTTP2Connection() (*HTTP2Connection, net.Conn) {
edgeConn, cfdConn := net.Pipe()
var connIndex = uint8(0)
connIndex := uint8(0)
log := zerolog.Nop()
obs := NewObserver(&log, &log)
controlStream := NewControlStream(
@@ -51,7 +50,7 @@ func newTestHTTP2Connection() (*HTTP2Connection, net.Conn) {
cfdConn,
// OriginProxy is set in testConfigManager
testOrchestrator,
&pogs.ConnectionOptions{},
&client.ConnectionOptionsSnapshot{},
obs,
connIndex,
controlStream,
@@ -62,7 +61,7 @@ func newTestHTTP2Connection() (*HTTP2Connection, net.Conn) {
func TestHTTP2ConfigurationSet(t *testing.T) {
http2Conn, edgeConn := newTestHTTP2Connection()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
var wg sync.WaitGroup
wg.Add(1)
go func() {
@@ -74,7 +73,7 @@ func TestHTTP2ConfigurationSet(t *testing.T) {
require.NoError(t, err)
reqBody := []byte(`{
"version": 2,
"version": 2,
"config": {"warp-routing": {"enabled": true}, "originRequest" : {"connectTimeout": 10}, "ingress" : [ {"hostname": "test", "service": "https://localhost:8000" } , {"service": "http_status:404"} ]}}
`)
reader := bytes.NewReader(reqBody)
@@ -130,7 +129,7 @@ func TestServeHTTP(t *testing.T) {
http2Conn, edgeConn := newTestHTTP2Connection()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
var wg sync.WaitGroup
wg.Add(1)
go func() {
@@ -261,7 +260,7 @@ func (w *wsRespWriter) close() {
func TestServeWS(t *testing.T) {
http2Conn, _ := newTestHTTP2Connection()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
respWriter := newWSRespWriter()
readPipe, writePipe := io.Pipe()
@@ -301,7 +300,7 @@ func TestServeWS(t *testing.T) {
func TestNoWriteAfterServeHTTPReturns(t *testing.T) {
cfdHTTP2Conn, edgeTCPConn := newTestHTTP2Connection()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
var wg sync.WaitGroup
serverDone := make(chan struct{})
@@ -379,7 +378,7 @@ func TestServeControlStream(t *testing.T) {
)
http2Conn.controlStreamHandler = controlStream
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
var wg sync.WaitGroup
wg.Add(1)
go func() {
@@ -433,7 +432,7 @@ func TestFailRegistration(t *testing.T) {
)
http2Conn.controlStreamHandler = controlStream
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
var wg sync.WaitGroup
wg.Add(1)
go func() {
@@ -484,7 +483,7 @@ func TestGracefulShutdownHTTP2(t *testing.T) {
http2Conn.controlStreamHandler = controlStream
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
var wg sync.WaitGroup
wg.Add(1)
go func() {
@@ -534,7 +533,7 @@ func TestGracefulShutdownHTTP2(t *testing.T) {
}
func TestServeTCP_RateLimited(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
http2Conn, edgeConn := newTestHTTP2Connection()
var wg sync.WaitGroup
@@ -566,7 +565,7 @@ func TestServeTCP_RateLimited(t *testing.T) {
func benchmarkServeHTTP(b *testing.B, test testRequest) {
http2Conn, edgeConn := newTestHTTP2Connection()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(b.Context())
var wg sync.WaitGroup
wg.Add(1)
go func() {
+5 -5
View File
@@ -17,12 +17,12 @@ import (
"github.com/rs/zerolog"
"golang.org/x/sync/errgroup"
"github.com/cloudflare/cloudflared/client"
cfdflow "github.com/cloudflare/cloudflared/flow"
cfdquic "github.com/cloudflare/cloudflared/quic"
"github.com/cloudflare/cloudflared/tracing"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
rpcquic "github.com/cloudflare/cloudflared/tunnelrpc/quic"
)
@@ -44,7 +44,7 @@ type quicConnection struct {
orchestrator Orchestrator
datagramHandler DatagramSessionHandler
controlStreamHandler ControlStreamHandler
connOptions *tunnelpogs.ConnectionOptions
connOptions *client.ConnectionOptionsSnapshot
connIndex uint8
rpcTimeout time.Duration
@@ -60,7 +60,7 @@ func NewTunnelConnection(
orchestrator Orchestrator,
datagramSessionHandler DatagramSessionHandler,
controlStreamHandler ControlStreamHandler,
connOptions *pogs.ConnectionOptions,
connOptions *client.ConnectionOptionsSnapshot,
rpcTimeout time.Duration,
streamWriteTimeout time.Duration,
gracePeriod time.Duration,
@@ -131,7 +131,7 @@ func (q *quicConnection) Serve(ctx context.Context) error {
// serveControlStream will serve the RPC; blocking until the control plane is done.
func (q *quicConnection) serveControlStream(ctx context.Context, controlStream quic.Stream) error {
return q.controlStreamHandler.ServeControlStream(ctx, controlStream, q.connOptions, q.orchestrator)
return q.controlStreamHandler.ServeControlStream(ctx, controlStream, q.connOptions.ConnectionOptions(), q.orchestrator)
}
// Close the connection with no errors specified.
@@ -235,7 +235,7 @@ func (q *quicConnection) dispatchRequest(ctx context.Context, stream *rpcquic.Re
}
// UpdateConfiguration is the RPC method invoked by edge when there is a new configuration
func (q *quicConnection) UpdateConfiguration(ctx context.Context, version int32, config []byte) *tunnelpogs.UpdateConfigurationResponse {
func (q *quicConnection) UpdateConfiguration(ctx context.Context, version int32, config []byte) *pogs.UpdateConfigurationResponse {
return q.orchestrator.UpdateConfig(version, config)
}
+24 -13
View File
@@ -29,6 +29,8 @@ import (
"github.com/stretchr/testify/require"
"golang.org/x/net/nettest"
"github.com/cloudflare/cloudflared/client"
"github.com/cloudflare/cloudflared/config"
cfdflow "github.com/cloudflare/cloudflared/flow"
"github.com/cloudflare/cloudflared/datagramsession"
@@ -59,7 +61,7 @@ func TestQUICServer(t *testing.T) {
err := wsutil.WriteClientBinary(wsBuf, []byte("Hello"))
require.NoError(t, err)
var tests = []struct {
tests := []struct {
desc string
dest string
connectionType pogs.ConnectionType
@@ -149,7 +151,7 @@ func TestQUICServer(t *testing.T) {
for i, test := range tests {
test := test // capture range variable
t.Run(test.desc, func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
// Start a UDP Listener for QUIC.
udpAddr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0")
require.NoError(t, err)
@@ -193,6 +195,7 @@ func (fakeControlStream) ServeControlStream(ctx context.Context, rw io.ReadWrite
<-ctx.Done()
return nil
}
func (fakeControlStream) IsStopped() bool {
return true
}
@@ -210,7 +213,7 @@ func quicServer(
session, err := listener.Accept(ctx)
require.NoError(t, err)
quicStream, err := session.OpenStreamSync(context.Background())
quicStream, err := session.OpenStreamSync(t.Context())
require.NoError(t, err)
stream := cfdquic.NewSafeStreamCloser(quicStream, defaultQUICTimeout, &log)
@@ -277,7 +280,7 @@ func (moc *mockOriginProxyWithRequest) ProxyHTTP(w ResponseWriter, tr *tracing.T
}
func TestBuildHTTPRequest(t *testing.T) {
var tests = []struct {
tests := []struct {
name string
connectRequest *pogs.ConnectRequest
body io.ReadCloser
@@ -498,7 +501,7 @@ func TestBuildHTTPRequest(t *testing.T) {
for _, test := range tests {
test := test // capture range variable
t.Run(test.name, func(t *testing.T) {
req, err := buildHTTPRequest(context.Background(), test.connectRequest, test.body, 0, &log)
req, err := buildHTTPRequest(t.Context(), test.connectRequest, test.body, 0, &log)
require.NoError(t, err)
test.req = test.req.WithContext(req.Context())
require.Equal(t, test.req, req.Request)
@@ -524,7 +527,7 @@ func TestServeUDPSession(t *testing.T) {
require.NoError(t, err)
defer udpListener.Close()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
// Establish QUIC connection with edge
edgeQUICSessionChan := make(chan quic.Connection)
@@ -605,7 +608,7 @@ func TestCreateUDPConnReuseSourcePort(t *testing.T) {
// TestTCPProxy_FlowRateLimited tests if the pogs.ConnectResponse returns the expected error and metadata, when a
// new flow is rate limited.
func TestTCPProxy_FlowRateLimited(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
// Start a UDP Listener for QUIC.
udpAddr, err := net.ResolveUDPAddr("udp", "127.0.0.1:0")
@@ -626,7 +629,7 @@ func TestTCPProxy_FlowRateLimited(t *testing.T) {
session, err := quicListener.Accept(ctx)
assert.NoError(t, err)
quicStream, err := session.OpenStreamSync(context.Background())
quicStream, err := session.OpenStreamSync(t.Context())
assert.NoError(t, err)
stream := cfdquic.NewSafeStreamCloser(quicStream, defaultQUICTimeout, &log)
@@ -687,9 +690,7 @@ func testCreateUDPConnReuseSourcePortForEdgeIP(t *testing.T, edgeIP netip.AddrPo
}
func serveSession(ctx context.Context, datagramConn *datagramV2Connection, edgeQUICSession quic.Connection, closeType closeReason, expectedReason string, t *testing.T) {
var (
payload = []byte(t.Name())
)
payload := []byte(t.Name())
sessionID := uuid.New()
cfdConn, originConn := net.Pipe()
// Registers and run a new session
@@ -802,7 +803,7 @@ func testTunnelConnection(t *testing.T, serverAddr netip.AddrPort, index uint8)
}
// Start a mock httpProxy
log := zerolog.New(io.Discard)
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
// Dial the QUIC connection to the edge
@@ -823,6 +824,15 @@ func testTunnelConnection(t *testing.T, serverAddr netip.AddrPort, index uint8)
sessionManager := datagramsession.NewManager(&log, datagramMuxer.SendToSession, sessionDemuxChan)
var connIndex uint8 = 0
packetRouter := ingress.NewPacketRouter(nil, datagramMuxer, connIndex, &log)
testDefaultDialer := ingress.NewDialer(ingress.WarpRoutingConfig{
ConnectTimeout: config.CustomDuration{Duration: 1 * time.Second},
TCPKeepAlive: config.CustomDuration{Duration: 15 * time.Second},
MaxActiveFlows: 0,
})
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &log)
datagramConn := &datagramV2Connection{
conn,
@@ -830,6 +840,7 @@ func testTunnelConnection(t *testing.T, serverAddr netip.AddrPort, index uint8)
sessionManager,
cfdflow.NewLimiter(0),
datagramMuxer,
originDialer,
packetRouter,
15 * time.Second,
0 * time.Second,
@@ -843,7 +854,7 @@ func testTunnelConnection(t *testing.T, serverAddr netip.AddrPort, index uint8)
&mockOrchestrator{originProxy: &mockOriginProxyWithRequest{}},
datagramConn,
fakeControlStream{},
&pogs.ConnectionOptions{},
&client.ConnectionOptionsSnapshot{},
15*time.Second,
0*time.Second,
0*time.Second,
+31 -3
View File
@@ -4,9 +4,11 @@ import (
"context"
"fmt"
"net"
"net/netip"
"time"
"github.com/google/uuid"
"github.com/pkg/errors"
pkgerrors "github.com/pkg/errors"
"github.com/quic-go/quic-go"
"github.com/rs/zerolog"
@@ -32,6 +34,10 @@ const (
demuxChanCapacity = 16
)
var (
errInvalidDestinationIP = errors.New("unable to parse destination IP")
)
// DatagramSessionHandler is a service that can serve datagrams for a connection and handle sessions from incoming
// connection streams.
type DatagramSessionHandler interface {
@@ -51,7 +57,10 @@ type datagramV2Connection struct {
// datagramMuxer mux/demux datagrams from quic connection
datagramMuxer *cfdquic.DatagramMuxerV2
packetRouter *ingress.PacketRouter
// originDialer is the origin dialer for UDP requests
originDialer ingress.OriginUDPDialer
// packetRouter acts as the origin router for ICMP requests
packetRouter *ingress.PacketRouter
rpcTimeout time.Duration
streamWriteTimeout time.Duration
@@ -61,6 +70,7 @@ type datagramV2Connection struct {
func NewDatagramV2Connection(ctx context.Context,
conn quic.Connection,
originDialer ingress.OriginUDPDialer,
icmpRouter ingress.ICMPRouter,
index uint8,
rpcTimeout time.Duration,
@@ -79,6 +89,7 @@ func NewDatagramV2Connection(ctx context.Context,
sessionManager: sessionManager,
flowLimiter: flowLimiter,
datagramMuxer: datagramMuxer,
originDialer: originDialer,
packetRouter: packetRouter,
rpcTimeout: rpcTimeout,
streamWriteTimeout: streamWriteTimeout,
@@ -128,12 +139,29 @@ func (q *datagramV2Connection) RegisterUdpSession(ctx context.Context, sessionID
tracing.EndWithErrorStatus(registerSpan, err)
return nil, err
}
// We need to force the net.IP to IPv4 (if it's an IPv4 address) otherwise the net.IP conversion from capnp
// will be a IPv4-mapped-IPv6 address.
// In the case that the address is IPv6 we leave it untouched and parse it as normal.
ip := dstIP.To4()
if ip == nil {
ip = dstIP
}
// Parse the dstIP and dstPort into a netip.AddrPort
// This should never fail because the IP was already parsed as a valid net.IP
destAddr, ok := netip.AddrFromSlice(ip)
if !ok {
log.Err(errInvalidDestinationIP).Msgf("Failed to parse destination proxy IP: %s", ip)
tracing.EndWithErrorStatus(registerSpan, errInvalidDestinationIP)
q.flowLimiter.Release()
return nil, errInvalidDestinationIP
}
dstAddrPort := netip.AddrPortFrom(destAddr, dstPort)
// Each session is a series of datagram from an eyeball to a dstIP:dstPort.
// (src port, dst IP, dst port) uniquely identifies a session, so it needs a dedicated connected socket.
originProxy, err := ingress.DialUDP(dstIP, dstPort)
originProxy, err := q.originDialer.DialUDP(dstAddrPort)
if err != nil {
log.Err(err).Msgf("Failed to create udp proxy to %s:%d", dstIP, dstPort)
log.Err(err).Msgf("Failed to create udp proxy to %s", dstAddrPort)
tracing.EndWithErrorStatus(registerSpan, err)
q.flowLimiter.Release()
return nil, err
+8 -4
View File
@@ -16,8 +16,7 @@ import (
"github.com/cloudflare/cloudflared/mocks"
)
type mockQuicConnection struct {
}
type mockQuicConnection struct{}
func (m *mockQuicConnection) AcceptStream(_ context.Context) (quic.Stream, error) {
return nil, nil
@@ -71,6 +70,10 @@ func (m *mockQuicConnection) ReceiveDatagram(_ context.Context) ([]byte, error)
return nil, nil
}
func (m *mockQuicConnection) AddPath(*quic.Transport) (*quic.Path, error) {
return nil, nil
}
func TestRateLimitOnNewDatagramV2UDPSession(t *testing.T) {
log := zerolog.Nop()
conn := &mockQuicConnection{}
@@ -78,9 +81,10 @@ func TestRateLimitOnNewDatagramV2UDPSession(t *testing.T) {
flowLimiterMock := mocks.NewMockLimiter(ctrl)
datagramConn := NewDatagramV2Connection(
context.Background(),
t.Context(),
conn,
nil,
nil,
0,
0*time.Second,
0*time.Second,
@@ -91,6 +95,6 @@ func TestRateLimitOnNewDatagramV2UDPSession(t *testing.T) {
flowLimiterMock.EXPECT().Acquire("udp").Return(cfdflow.ErrTooManyActiveFlows)
flowLimiterMock.EXPECT().Release().Times(0)
_, err := datagramConn.RegisterUdpSession(context.Background(), uuid.New(), net.IPv4(0, 0, 0, 0), 1000, 1*time.Second, "")
_, err := datagramConn.RegisterUdpSession(t.Context(), uuid.New(), net.IPv4(0, 0, 0, 0), 1000, 1*time.Second, "")
require.ErrorIs(t, err, cfdflow.ErrTooManyActiveFlows)
}
+15 -4
View File
@@ -2,11 +2,11 @@ package connection
import (
"context"
"fmt"
"net"
"time"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/quic-go/quic-go"
"github.com/rs/zerolog"
@@ -16,10 +16,17 @@ import (
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
var (
ErrUnsupportedRPCUDPRegistration = errors.New("datagram v3 does not support RegisterUdpSession RPC")
ErrUnsupportedRPCUDPUnregistration = errors.New("datagram v3 does not support UnregisterUdpSession RPC")
)
type datagramV3Connection struct {
conn quic.Connection
conn quic.Connection
index uint8
// datagramMuxer mux/demux datagrams from quic connection
datagramMuxer cfdquic.DatagramConn
metrics cfdquic.Metrics
logger *zerolog.Logger
}
@@ -40,7 +47,9 @@ func NewDatagramV3Connection(ctx context.Context,
return &datagramV3Connection{
conn,
index,
datagramMuxer,
metrics,
logger,
}
}
@@ -50,9 +59,11 @@ func (d *datagramV3Connection) Serve(ctx context.Context) error {
}
func (d *datagramV3Connection) RegisterUdpSession(ctx context.Context, sessionID uuid.UUID, dstIP net.IP, dstPort uint16, closeAfterIdleHint time.Duration, traceContext string) (*pogs.RegisterUdpSessionResponse, error) {
return nil, fmt.Errorf("datagram v3 does not support RegisterUdpSession RPC")
d.metrics.UnsupportedRemoteCommand(d.index, "register_udp_session")
return nil, ErrUnsupportedRPCUDPRegistration
}
func (d *datagramV3Connection) UnregisterUdpSession(ctx context.Context, sessionID uuid.UUID, message string) error {
return fmt.Errorf("datagram v3 does not support UnregisterUdpSession RPC")
d.metrics.UnsupportedRemoteCommand(d.index, "unregister_udp_session")
return ErrUnsupportedRPCUDPUnregistration
}
+1 -1
View File
@@ -84,7 +84,7 @@ func (s *Session) waitForCloseCondition(ctx context.Context, closeAfterIdle time
// Closing dstConn cancels read so dstToTransport routine in Serve() can return
defer s.dstConn.Close()
if closeAfterIdle == 0 {
// provide deafult is caller doesn't specify one
// provide default is caller doesn't specify one
closeAfterIdle = defaultCloseIdleAfter
}
+19 -18
View File
@@ -12,6 +12,7 @@ import (
"github.com/google/uuid"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"golang.org/x/sync/errgroup"
@@ -34,12 +35,10 @@ func TestCloseIdle(t *testing.T) {
}
func testSessionReturns(t *testing.T, closeBy closeMethod, closeAfterIdle time.Duration) {
var (
localCloseReason = &errClosedSession{
message: "connection closed by origin",
byRemote: false,
}
)
localCloseReason := &errClosedSession{
message: "connection closed by origin",
byRemote: false,
}
sessionID := uuid.New()
cfdConn, originConn := net.Pipe()
payload := testPayload(sessionID)
@@ -48,28 +47,28 @@ func testSessionReturns(t *testing.T, closeBy closeMethod, closeAfterIdle time.D
mg := NewManager(&log, nil, nil)
session := mg.newSession(sessionID, cfdConn)
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
sessionDone := make(chan struct{})
go func() {
closedByRemote, err := session.Serve(ctx, closeAfterIdle)
switch closeBy {
case closeByContext:
require.Equal(t, context.Canceled, err)
require.False(t, closedByRemote)
assert.Equal(t, context.Canceled, err)
assert.False(t, closedByRemote)
case closeByCallingClose:
require.Equal(t, localCloseReason, err)
require.Equal(t, localCloseReason.byRemote, closedByRemote)
assert.Equal(t, localCloseReason, err)
assert.Equal(t, localCloseReason.byRemote, closedByRemote)
case closeByTimeout:
require.Equal(t, SessionIdleErr(closeAfterIdle), err)
require.False(t, closedByRemote)
assert.Equal(t, SessionIdleErr(closeAfterIdle), err)
assert.False(t, closedByRemote)
}
close(sessionDone)
}()
go func() {
n, err := session.transportToDst(payload)
require.NoError(t, err)
require.Equal(t, len(payload), n)
assert.NoError(t, err)
assert.Equal(t, len(payload), n)
}()
readBuffer := make([]byte, len(payload)+1)
@@ -84,6 +83,8 @@ func testSessionReturns(t *testing.T, closeBy closeMethod, closeAfterIdle time.D
cancel()
case closeByCallingClose:
session.close(localCloseReason)
default:
// ignore
}
<-sessionDone
@@ -125,10 +126,10 @@ func testActiveSessionNotClosed(t *testing.T, readFromDst bool, writeToDst bool)
startTime := time.Now()
activeUntil := startTime.Add(activeTime)
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
errGroup, ctx := errgroup.WithContext(ctx)
errGroup.Go(func() error {
session.Serve(ctx, closeAfterIdle)
_, _ = session.Serve(ctx, closeAfterIdle)
if time.Now().Before(startTime.Add(activeTime)) {
return fmt.Errorf("session closed while it's still active")
}
@@ -208,7 +209,7 @@ func TestZeroBytePayload(t *testing.T) {
mg := NewManager(&nopLogger, sender.muxSession, nil)
session := mg.newSession(sessionID, cfdConn)
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
errGroup, ctx := errgroup.WithContext(ctx)
errGroup.Go(func() error {
// Read from underlying conn and send to transport
-10
View File
@@ -1,10 +0,0 @@
FROM golang:1.22.10 as builder
ENV GO111MODULE=on \
CGO_ENABLED=0
WORKDIR /go/src/github.com/cloudflare/cloudflared/
RUN apt-get update
COPY . .
RUN .teamcity/install-cloudflare-go.sh
# compile cloudflared
RUN PATH="/tmp/go/bin:$PATH" make cloudflared
RUN cp /go/src/github.com/cloudflare/cloudflared/cloudflared /usr/local/bin/
+29 -7
View File
@@ -1,5 +1,7 @@
package features
import "slices"
const (
FeatureSerializedHeaders = "serialized_headers"
FeatureQuickReconnects = "quick_reconnects"
@@ -8,7 +10,9 @@ const (
FeaturePostQuantum = "postquantum"
FeatureQUICSupportEOF = "support_quic_eof"
FeatureManagementLogs = "management_logs"
FeatureDatagramV3 = "support_datagram_v3"
FeatureDatagramV3_1 = "support_datagram_v3_1"
DeprecatedFeatureDatagramV3 = "support_datagram_v3" // Deprecated: TUN-9291
)
var defaultFeatures = []string{
@@ -19,11 +23,25 @@ var defaultFeatures = []string{
FeatureManagementLogs,
}
// List of features that are no longer in-use.
var deprecatedFeatures = []string{
DeprecatedFeatureDatagramV3,
}
// Features set by user provided flags
type staticFeatures struct {
PostQuantumMode *PostQuantumMode
}
type FeatureSnapshot struct {
PostQuantum PostQuantumMode
DatagramVersion DatagramVersion
// We provide the list of features since we need it to send in the ConnectionOptions during connection
// registrations.
FeaturesList []string
}
type PostQuantumMode uint8
const (
@@ -40,15 +58,19 @@ const (
// DatagramV2 is the currently supported datagram protocol for UDP and ICMP packets
DatagramV2 DatagramVersion = FeatureDatagramV2
// DatagramV3 is a new datagram protocol for UDP and ICMP packets. It is not backwards compatible with datagram v2.
DatagramV3 DatagramVersion = FeatureDatagramV3
DatagramV3 DatagramVersion = FeatureDatagramV3_1
)
// Remove any duplicates from the slice
func Dedup(slice []string) []string {
// Remove any duplicate features from the list and remove deprecated features
func dedupAndRemoveFeatures(features []string) []string {
// Convert the slice into a set
set := make(map[string]bool, 0)
for _, str := range slice {
set[str] = true
set := map[string]bool{}
for _, feature := range features {
// Remove deprecated features from the provided list
if slices.Contains(deprecatedFeatures, feature) {
continue
}
set[feature] = true
}
// Convert the set back into a slice
+57 -46
View File
@@ -15,28 +15,31 @@ import (
const (
featureSelectorHostname = "cfd-features.argotunnel.com"
defaultRefreshFreq = time.Hour * 6
lookupTimeout = time.Second * 10
defaultLookupFreq = time.Hour
)
// If the TXT record adds other fields, the umarshal logic will ignore those keys
// If the TXT record is missing a key, the field will unmarshal to the default Go value
type featuresRecord struct {
// support_datagram_v3
DatagramV3Percentage int32 `json:"dv3"`
DatagramV3Percentage uint32 `json:"dv3_1"`
// DatagramV3Percentage int32 `json:"dv3"` // Removed in TUN-9291
// PostQuantumPercentage int32 `json:"pq"` // Removed in TUN-7970
}
func NewFeatureSelector(ctx context.Context, accountTag string, cliFeatures []string, pq bool, logger *zerolog.Logger) (*FeatureSelector, error) {
return newFeatureSelector(ctx, accountTag, logger, newDNSResolver(), cliFeatures, pq, defaultRefreshFreq)
func NewFeatureSelector(ctx context.Context, accountTag string, cliFeatures []string, pq bool, logger *zerolog.Logger) (FeatureSelector, error) {
return newFeatureSelector(ctx, accountTag, logger, newDNSResolver(), cliFeatures, pq, defaultLookupFreq)
}
// FeatureSelector determines if this account will try new features. It periodically queries a DNS TXT record
// to see which features are turned on.
type FeatureSelector struct {
accountHash int32
type FeatureSelector interface {
Snapshot() FeatureSnapshot
}
// FeatureSelector determines if this account will try new features; loaded once during startup.
type featureSelector struct {
accountHash uint32
logger *zerolog.Logger
resolver resolver
@@ -44,11 +47,11 @@ type FeatureSelector struct {
cliFeatures []string
// lock protects concurrent access to dynamic features
lock sync.RWMutex
features featuresRecord
lock sync.RWMutex
remoteFeatures featuresRecord
}
func newFeatureSelector(ctx context.Context, accountTag string, logger *zerolog.Logger, resolver resolver, cliFeatures []string, pq bool, refreshFreq time.Duration) (*FeatureSelector, error) {
func newFeatureSelector(ctx context.Context, accountTag string, logger *zerolog.Logger, resolver resolver, cliFeatures []string, pq bool, refreshFreq time.Duration) (*featureSelector, error) {
// Combine default features and user-provided features
var pqMode *PostQuantumMode
if pq {
@@ -59,28 +62,40 @@ func newFeatureSelector(ctx context.Context, accountTag string, logger *zerolog.
staticFeatures := staticFeatures{
PostQuantumMode: pqMode,
}
selector := &FeatureSelector{
selector := &featureSelector{
accountHash: switchThreshold(accountTag),
logger: logger,
resolver: resolver,
staticFeatures: staticFeatures,
cliFeatures: Dedup(cliFeatures),
cliFeatures: dedupAndRemoveFeatures(cliFeatures),
}
// Load the remote features
if err := selector.refresh(ctx); err != nil {
logger.Err(err).Msg("Failed to fetch features, default to disable")
}
// Spin off reloading routine
go selector.refreshLoop(ctx, refreshFreq)
return selector, nil
}
func (fs *FeatureSelector) accountEnabled(percentage int32) bool {
func (fs *featureSelector) Snapshot() FeatureSnapshot {
fs.lock.RLock()
defer fs.lock.RUnlock()
return FeatureSnapshot{
PostQuantum: fs.postQuantumMode(),
DatagramVersion: fs.datagramVersion(),
FeaturesList: fs.clientFeatures(),
}
}
func (fs *featureSelector) accountEnabled(percentage uint32) bool {
return percentage > fs.accountHash
}
func (fs *FeatureSelector) PostQuantumMode() PostQuantumMode {
func (fs *featureSelector) postQuantumMode() PostQuantumMode {
if fs.staticFeatures.PostQuantumMode != nil {
return *fs.staticFeatures.PostQuantumMode
}
@@ -88,12 +103,9 @@ func (fs *FeatureSelector) PostQuantumMode() PostQuantumMode {
return PostQuantumPrefer
}
func (fs *FeatureSelector) DatagramVersion() DatagramVersion {
fs.lock.RLock()
defer fs.lock.RUnlock()
func (fs *featureSelector) datagramVersion() DatagramVersion {
// If user provides the feature via the cli, we take it as priority over remote feature evaluation
if slices.Contains(fs.cliFeatures, FeatureDatagramV3) {
if slices.Contains(fs.cliFeatures, FeatureDatagramV3_1) {
return DatagramV3
}
// If the user specifies DatagramV2, we also take that over remote
@@ -101,36 +113,20 @@ func (fs *FeatureSelector) DatagramVersion() DatagramVersion {
return DatagramV2
}
if fs.accountEnabled(fs.features.DatagramV3Percentage) {
if fs.accountEnabled(fs.remoteFeatures.DatagramV3Percentage) {
return DatagramV3
}
return DatagramV2
}
// ClientFeatures will return the list of currently available features that cloudflared should provide to the edge.
//
// This list is dynamic and can change in-between returns.
func (fs *FeatureSelector) ClientFeatures() []string {
// clientFeatures will return the list of currently available features that cloudflared should provide to the edge.
func (fs *featureSelector) clientFeatures() []string {
// Evaluate any remote features along with static feature list to construct the list of features
return Dedup(slices.Concat(defaultFeatures, fs.cliFeatures, []string{string(fs.DatagramVersion())}))
return dedupAndRemoveFeatures(slices.Concat(defaultFeatures, fs.cliFeatures, []string{string(fs.datagramVersion())}))
}
func (fs *FeatureSelector) refreshLoop(ctx context.Context, refreshFreq time.Duration) {
ticker := time.NewTicker(refreshFreq)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
err := fs.refresh(ctx)
if err != nil {
fs.logger.Err(err).Msg("Failed to refresh feature selector")
}
}
}
}
func (fs *FeatureSelector) refresh(ctx context.Context) error {
func (fs *featureSelector) refresh(ctx context.Context) error {
record, err := fs.resolver.lookupRecord(ctx)
if err != nil {
return err
@@ -144,11 +140,26 @@ func (fs *FeatureSelector) refresh(ctx context.Context) error {
fs.lock.Lock()
defer fs.lock.Unlock()
fs.features = features
fs.remoteFeatures = features
return nil
}
func (fs *featureSelector) refreshLoop(ctx context.Context, refreshFreq time.Duration) {
ticker := time.NewTicker(refreshFreq)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
err := fs.refresh(ctx)
if err != nil {
fs.logger.Err(err).Msg("Failed to refresh feature selector")
}
}
}
}
// resolver represents an object that can look up featuresRecord
type resolver interface {
lookupRecord(ctx context.Context) ([]byte, error)
@@ -180,8 +191,8 @@ func (dr *dnsResolver) lookupRecord(ctx context.Context) ([]byte, error) {
return []byte(records[0]), nil
}
func switchThreshold(accountTag string) int32 {
func switchThreshold(accountTag string) uint32 {
h := fnv.New32a()
_, _ = h.Write([]byte(accountTag))
return int32(h.Sum32() % 100)
return h.Sum32() % 100
}
+96 -64
View File
@@ -11,21 +11,26 @@ import (
"github.com/stretchr/testify/require"
)
const (
testAccountTag = "123456"
testAccountHash = 74 // switchThreshold of `accountTag`
)
func TestUnmarshalFeaturesRecord(t *testing.T) {
tests := []struct {
record []byte
expectedPercentage int32
expectedPercentage uint32
}{
{
record: []byte(`{"dv3":0}`),
record: []byte(`{"dv3_1":0}`),
expectedPercentage: 0,
},
{
record: []byte(`{"dv3":39}`),
record: []byte(`{"dv3_1":39}`),
expectedPercentage: 39,
},
{
record: []byte(`{"dv3":100}`),
record: []byte(`{"dv3_1":100}`),
expectedPercentage: 100,
},
{
@@ -34,6 +39,9 @@ func TestUnmarshalFeaturesRecord(t *testing.T) {
{
record: []byte(`{"kyber":768}`), // Unmarshal to default struct if key is not present
},
{
record: []byte(`{"pq": 101,"dv3":100}`), // Expired keys don't unmarshal to anything
},
}
for _, test := range tests {
@@ -61,7 +69,7 @@ func TestFeaturePrecedenceEvaluationPostQuantum(t *testing.T) {
{
name: "user_specified",
cli: true,
expectedFeatures: Dedup(append(defaultFeatures, FeaturePostQuantum)),
expectedFeatures: dedupAndRemoveFeatures(append(defaultFeatures, FeaturePostQuantum)),
expectedVersion: PostQuantumStrict,
},
}
@@ -69,10 +77,11 @@ func TestFeaturePrecedenceEvaluationPostQuantum(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
resolver := &staticResolver{record: featuresRecord{}}
selector, err := newFeatureSelector(context.Background(), test.name, &logger, resolver, []string{}, test.cli, time.Second)
selector, err := newFeatureSelector(t.Context(), test.name, &logger, resolver, []string{}, test.cli, time.Second)
require.NoError(t, err)
require.ElementsMatch(t, test.expectedFeatures, selector.ClientFeatures())
require.Equal(t, test.expectedVersion, selector.PostQuantumMode())
snapshot := selector.Snapshot()
require.ElementsMatch(t, test.expectedFeatures, snapshot.FeaturesList)
require.Equal(t, test.expectedVersion, snapshot.PostQuantum)
})
}
}
@@ -102,97 +111,120 @@ func TestFeaturePrecedenceEvaluationDatagramVersion(t *testing.T) {
},
{
name: "user_specified_v3",
cli: []string{FeatureDatagramV3},
cli: []string{FeatureDatagramV3_1},
remote: featuresRecord{},
expectedFeatures: Dedup(append(defaultFeatures, FeatureDatagramV3)),
expectedVersion: FeatureDatagramV3,
},
{
name: "remote_specified_v3",
cli: []string{},
remote: featuresRecord{
DatagramV3Percentage: 100,
},
expectedFeatures: Dedup(append(defaultFeatures, FeatureDatagramV3)),
expectedVersion: FeatureDatagramV3,
},
{
name: "remote_and_user_specified_v3",
cli: []string{FeatureDatagramV3},
remote: featuresRecord{
DatagramV3Percentage: 100,
},
expectedFeatures: Dedup(append(defaultFeatures, FeatureDatagramV3)),
expectedVersion: FeatureDatagramV3,
},
{
name: "remote_v3_and_user_specified_v2",
cli: []string{FeatureDatagramV2},
remote: featuresRecord{
DatagramV3Percentage: 100,
},
expectedFeatures: defaultFeatures,
expectedVersion: DatagramV2,
expectedFeatures: dedupAndRemoveFeatures(append(defaultFeatures, FeatureDatagramV3_1)),
expectedVersion: FeatureDatagramV3_1,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
resolver := &staticResolver{record: test.remote}
selector, err := newFeatureSelector(context.Background(), test.name, &logger, resolver, test.cli, false, time.Second)
selector, err := newFeatureSelector(t.Context(), test.name, &logger, resolver, test.cli, false, time.Second)
require.NoError(t, err)
require.ElementsMatch(t, test.expectedFeatures, selector.ClientFeatures())
require.Equal(t, test.expectedVersion, selector.DatagramVersion())
snapshot := selector.Snapshot()
require.ElementsMatch(t, test.expectedFeatures, snapshot.FeaturesList)
require.Equal(t, test.expectedVersion, snapshot.DatagramVersion)
})
}
}
func TestDeprecatedFeaturesRemoved(t *testing.T) {
logger := zerolog.Nop()
tests := []struct {
name string
cli []string
remote featuresRecord
expectedFeatures []string
}{
{
name: "no_removals",
cli: []string{},
remote: featuresRecord{},
expectedFeatures: defaultFeatures,
},
{
name: "support_datagram_v3",
cli: []string{DeprecatedFeatureDatagramV3},
remote: featuresRecord{},
expectedFeatures: defaultFeatures,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
resolver := &staticResolver{record: test.remote}
selector, err := newFeatureSelector(t.Context(), test.name, &logger, resolver, test.cli, false, time.Second)
require.NoError(t, err)
snapshot := selector.Snapshot()
require.ElementsMatch(t, test.expectedFeatures, snapshot.FeaturesList)
})
}
}
func TestRefreshFeaturesRecord(t *testing.T) {
// The hash of the accountTag is 82
accountTag := t.Name()
threshold := switchThreshold(accountTag)
percentages := []int32{0, 10, 81, 82, 83, 100, 101, 1000}
refreshFreq := time.Millisecond * 10
selector := newTestSelector(t, percentages, false, refreshFreq)
percentages := []uint32{0, 10, testAccountHash - 1, testAccountHash, testAccountHash + 1, 100, 101, 1000}
selector := newTestSelector(t, percentages, false, time.Minute)
// Starting out should default to DatagramV2
require.Equal(t, DatagramV2, selector.DatagramVersion())
snapshot := selector.Snapshot()
require.Equal(t, DatagramV2, snapshot.DatagramVersion)
for _, percentage := range percentages {
if percentage > threshold {
require.Equal(t, DatagramV3, selector.DatagramVersion())
snapshot = selector.Snapshot()
if percentage > testAccountHash {
require.Equal(t, DatagramV3, snapshot.DatagramVersion)
} else {
require.Equal(t, DatagramV2, selector.DatagramVersion())
require.Equal(t, DatagramV2, snapshot.DatagramVersion)
}
time.Sleep(refreshFreq + time.Millisecond)
// Manually progress the next refresh
_ = selector.refresh(t.Context())
}
// Make sure error doesn't override the last fetched features
require.Equal(t, DatagramV3, selector.DatagramVersion())
// Make sure a resolver error doesn't override the last fetched features
snapshot = selector.Snapshot()
require.Equal(t, DatagramV3, snapshot.DatagramVersion)
}
func TestSnapshotIsolation(t *testing.T) {
percentages := []uint32{testAccountHash, testAccountHash + 1}
selector := newTestSelector(t, percentages, false, time.Minute)
// Starting out should default to DatagramV2
snapshot := selector.Snapshot()
require.Equal(t, DatagramV2, snapshot.DatagramVersion)
// Manually progress the next refresh
_ = selector.refresh(t.Context())
snapshot2 := selector.Snapshot()
require.Equal(t, DatagramV3, snapshot2.DatagramVersion)
require.NotEqual(t, snapshot.DatagramVersion, snapshot2.DatagramVersion)
}
func TestStaticFeatures(t *testing.T) {
percentages := []int32{0}
percentages := []uint32{0}
// PostQuantum Enabled from user flag
selector := newTestSelector(t, percentages, true, time.Millisecond*10)
require.Equal(t, PostQuantumStrict, selector.PostQuantumMode())
selector := newTestSelector(t, percentages, true, time.Second)
snapshot := selector.Snapshot()
require.Equal(t, PostQuantumStrict, snapshot.PostQuantum)
// PostQuantum Disabled (or not set)
selector = newTestSelector(t, percentages, false, time.Millisecond*10)
require.Equal(t, PostQuantumPrefer, selector.PostQuantumMode())
selector = newTestSelector(t, percentages, false, time.Second)
snapshot = selector.Snapshot()
require.Equal(t, PostQuantumPrefer, snapshot.PostQuantum)
}
func newTestSelector(t *testing.T, percentages []int32, pq bool, refreshFreq time.Duration) *FeatureSelector {
accountTag := t.Name()
func newTestSelector(t *testing.T, percentages []uint32, pq bool, refreshFreq time.Duration) *featureSelector {
logger := zerolog.Nop()
resolver := &mockResolver{
percentages: percentages,
}
selector, err := newFeatureSelector(context.Background(), accountTag, &logger, resolver, []string{}, pq, refreshFreq)
selector, err := newFeatureSelector(t.Context(), testAccountTag, &logger, resolver, []string{}, pq, refreshFreq)
require.NoError(t, err)
return selector
@@ -200,7 +232,7 @@ func newTestSelector(t *testing.T, percentages []int32, pq bool, refreshFreq tim
type mockResolver struct {
nextIndex int
percentages []int32
percentages []uint32
}
func (mr *mockResolver) lookupRecord(ctx context.Context) ([]byte, error) {
+51 -46
View File
@@ -1,47 +1,47 @@
module github.com/cloudflare/cloudflared
go 1.22
go 1.24
require (
github.com/coredns/coredns v1.11.3
github.com/coredns/coredns v1.12.2
github.com/coreos/go-oidc/v3 v3.10.0
github.com/coreos/go-systemd/v22 v22.5.0
github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434
github.com/fortytw2/leaktest v1.3.0
github.com/fsnotify/fsnotify v1.4.9
github.com/getsentry/sentry-go v0.16.0
github.com/go-chi/chi/v5 v5.0.8
github.com/go-chi/chi/v5 v5.0.10
github.com/go-chi/cors v1.2.1
github.com/go-jose/go-jose/v4 v4.0.1
github.com/go-jose/go-jose/v4 v4.1.0
github.com/gobwas/ws v1.2.1
github.com/google/gopacket v1.1.19
github.com/google/uuid v1.6.0
github.com/gorilla/websocket v1.4.2
github.com/gorilla/websocket v1.5.0
github.com/json-iterator/go v1.1.12
github.com/mattn/go-colorable v0.1.13
github.com/miekg/dns v1.1.58
github.com/miekg/dns v1.1.66
github.com/mitchellh/go-homedir v1.1.0
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.19.1
github.com/prometheus/client_model v0.6.0
github.com/quic-go/quic-go v0.45.0
github.com/prometheus/client_golang v1.22.0
github.com/prometheus/client_model v0.6.2
github.com/quic-go/quic-go v0.52.0
github.com/rs/zerolog v1.20.0
github.com/stretchr/testify v1.9.0
github.com/stretchr/testify v1.10.0
github.com/urfave/cli/v2 v2.3.0
go.opentelemetry.io/contrib/propagators v0.22.0
go.opentelemetry.io/otel v1.26.0
go.opentelemetry.io/otel v1.35.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.26.0
go.opentelemetry.io/otel/sdk v1.26.0
go.opentelemetry.io/otel/trace v1.26.0
go.opentelemetry.io/otel/sdk v1.35.0
go.opentelemetry.io/otel/trace v1.35.0
go.opentelemetry.io/proto/otlp v1.2.0
go.uber.org/automaxprocs v1.4.0
go.uber.org/mock v0.5.0
golang.org/x/crypto v0.31.0
golang.org/x/net v0.26.0
golang.org/x/sync v0.10.0
golang.org/x/sys v0.28.0
golang.org/x/term v0.27.0
google.golang.org/protobuf v1.34.1
go.uber.org/automaxprocs v1.6.0
go.uber.org/mock v0.5.1
golang.org/x/crypto v0.38.0
golang.org/x/net v0.40.0
golang.org/x/sync v0.14.0
golang.org/x/sys v0.33.0
golang.org/x/term v0.32.0
google.golang.org/protobuf v1.36.6
gopkg.in/natefinch/lumberjack.v2 v2.0.0
gopkg.in/yaml.v3 v3.0.1
nhooyr.io/websocket v1.8.7
@@ -52,8 +52,9 @@ require (
github.com/BurntSushi/toml v1.2.0 // indirect
github.com/apparentlymart/go-cidr v1.1.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/coredns/caddy v1.1.1 // indirect
github.com/bytedance/sonic v1.12.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/coredns/caddy v1.1.2-0.20241029205200-8de985351a98 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51 // indirect
@@ -61,38 +62,42 @@ require (
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 // indirect
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870 // indirect
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/gin-gonic/gin v1.9.1 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
github.com/go-playground/validator/v10 v10.15.1 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/gobwas/httphead v0.1.0 // indirect
github.com/gobwas/pool v0.2.1 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/pprof v0.0.0-20230817174616-7a8ec2ada47b // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1 // indirect
github.com/google/pprof v0.0.0-20250418163039-24c5476c6587 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 // indirect
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645 // indirect
github.com/klauspost/compress v1.15.11 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/kylelemons/godebug v1.1.0 // indirect
github.com/mattn/go-isatty v0.0.16 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.5 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/onsi/ginkgo/v2 v2.13.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/onsi/ginkgo/v2 v2.23.4 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/common v0.53.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/pelletier/go-toml/v2 v2.0.9 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/common v0.64.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
go.opentelemetry.io/otel/metric v1.26.0 // indirect
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 // indirect
golang.org/x/mod v0.18.0 // indirect
golang.org/x/oauth2 v0.18.0 // indirect
golang.org/x/text v0.21.0 // indirect
golang.org/x/tools v0.22.0 // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240311132316-a219d84964c2 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237 // indirect
google.golang.org/grpc v1.63.2 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.35.0 // indirect
golang.org/x/arch v0.4.0 // indirect
golang.org/x/mod v0.24.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/text v0.25.0 // indirect
golang.org/x/tools v0.32.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250505200425-f936aa4a68b2 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250512202823-5a2f75b736a9 // indirect
google.golang.org/grpc v1.72.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
)
@@ -104,4 +109,4 @@ replace github.com/prometheus/golang_client => github.com/prometheus/golang_clie
replace gopkg.in/yaml.v3 => gopkg.in/yaml.v3 v3.0.1
// This fork is based on quic-go v0.45
replace github.com/quic-go/quic-go => github.com/chungthuang/quic-go v0.45.1-0.20250128102735-2687bd175910
replace github.com/quic-go/quic-go => github.com/chungthuang/quic-go v0.45.1-0.20250428085412-43229ad201fd
+138 -138
View File
@@ -5,14 +5,22 @@ github.com/apparentlymart/go-cidr v1.1.0 h1:2mAhrMoF+nhXqxTzSZMUzDHkLjmIHC+Zzn4t
github.com/apparentlymart/go-cidr v1.1.0/go.mod h1:EBcsNrHc3zQeuaeCeCtQruQm+n9/YjEn/vI25Lg7Gwc=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chungthuang/quic-go v0.45.1-0.20250128102735-2687bd175910 h1:/hTvBpxBDj/3NIzTodi1oEOyNBpirvgDSPKSV7VqAZU=
github.com/chungthuang/quic-go v0.45.1-0.20250128102735-2687bd175910/go.mod h1:1dLehS7TIR64+vxGR70GDcatWTOtMX2PUtnKsjbTurI=
github.com/coredns/caddy v1.1.1 h1:2eYKZT7i6yxIfGP3qLJoJ7HAsDJqYB+X68g4NYjSrE0=
github.com/coredns/caddy v1.1.1/go.mod h1:A6ntJQlAWuQfFlsd9hvigKbo2WS0VUs2l1e2F+BawD4=
github.com/coredns/coredns v1.11.3 h1:8RjnpZc42db5th84/QJKH2i137ecJdzZK1HJwhetSPk=
github.com/coredns/coredns v1.11.3/go.mod h1:lqFkDsHjEUdY7LJ75Nib3lwqJGip6ewWOqNIf8OavIQ=
github.com/bytedance/sonic v1.12.0 h1:YGPgxF9xzaCNvd/ZKdQ28yRovhfMFZQjuk6fKBzZ3ls=
github.com/bytedance/sonic v1.12.0/go.mod h1:B8Gt/XvtZ3Fqj+iSKMypzymZxw/FVwgIGKzMzT9r/rk=
github.com/bytedance/sonic/loader v0.2.0 h1:zNprn+lsIP06C/IqCHs3gPQIvnvpKbbxyXQP1iU4kWM=
github.com/bytedance/sonic/loader v0.2.0/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chungthuang/quic-go v0.45.1-0.20250428085412-43229ad201fd h1:VdYI5zFQ2h1/qzoC6rhyPx479bkF8i177Qpg4Q2n1vk=
github.com/chungthuang/quic-go v0.45.1-0.20250428085412-43229ad201fd/go.mod h1:MFlGGpcpJqRAfmYi6NC2cptDPSxRWTOGNuP4wqrWmzQ=
github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y=
github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w=
github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
github.com/coredns/caddy v1.1.2-0.20241029205200-8de985351a98 h1:c+Epklw9xk6BZ1OFBPWLA2PcL8QalKvl3if8CP9x8uw=
github.com/coredns/caddy v1.1.2-0.20241029205200-8de985351a98/go.mod h1:A6ntJQlAWuQfFlsd9hvigKbo2WS0VUs2l1e2F+BawD4=
github.com/coredns/coredns v1.12.2 h1:G4oDfi340zlVsriZ8nYiUemiQIew7nqOO+QPvPxIA4Y=
github.com/coredns/coredns v1.12.2/go.mod h1:GFz31oVOfCyMArFoypfu1SoaFoNkbdh6lDxtF1B6vfU=
github.com/coreos/go-oidc/v3 v3.10.0 h1:tDnXHnLyiTVyT/2zLDGj09pFPkhND8Gl8lnTRhoEaJU=
github.com/coreos/go-oidc/v3 v3.10.0/go.mod h1:5j11xcw0D3+SGxn6Z/WFADsgcWVMyNAlSQupk0KK3ac=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
@@ -21,7 +29,6 @@ github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSV
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/cpuguy83/go-md2man/v2 v2.0.0 h1:EoUDS0afbrsXAZ9YQ9jdu/mZ2sXgT1/2yyNng4PGlyM=
github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
@@ -42,38 +49,40 @@ github.com/fortytw2/leaktest v1.3.0 h1:u8491cBMTQ8ft8aeV+adlcytMZylmA5nnwwkRZjI8
github.com/fortytw2/leaktest v1.3.0/go.mod h1:jDsjWgpAGjm2CA7WthBh/CdZYEPF31XHquHwclZch5g=
github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/gabriel-vasile/mimetype v1.4.2 h1:w5qFW6JKBz9Y393Y4q372O9A7cUSequkh1Q7OhCmWKU=
github.com/gabriel-vasile/mimetype v1.4.2/go.mod h1:zApsH/mKG4w07erKIaJPFiX0Tsq9BFQgN3qGY5GnNgA=
github.com/getsentry/sentry-go v0.16.0 h1:owk+S+5XcgJLlGR/3+3s6N4d+uKwqYvh/eS0AIMjPWo=
github.com/getsentry/sentry-go v0.16.0/go.mod h1:ZXCloQLj0pG7mja5NK6NPf2V4A88YJ4pNlc2mOHwh6Y=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
github.com/gin-gonic/gin v1.6.3/go.mod h1:75u5sXoLsGZoRN5Sgbi1eraJ4GU3++wFwWzhwvtwp4M=
github.com/gin-gonic/gin v1.8.1 h1:4+fr/el88TOO3ewCmQr8cx/CtZ/umlIRIs5M4NTNjf8=
github.com/gin-gonic/gin v1.8.1/go.mod h1:ji8BvRH1azfM+SYow9zQ6SZMvR8qOMZHmsCuWR9tTTk=
github.com/go-chi/chi/v5 v5.0.8 h1:lD+NLqFcAi1ovnVZpsnObHGW4xb4J8lNmoYVfECH1Y0=
github.com/go-chi/chi/v5 v5.0.8/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/gin-gonic/gin v1.9.1 h1:4idEAncQnU5cB7BeOkPtxjfCSye0AAm1R0RVIqJ+Jmg=
github.com/gin-gonic/gin v1.9.1/go.mod h1:hPrL7YrpYKXt5YId3A/Tnip5kqbEAP+KLuI3SUcPTeU=
github.com/go-chi/chi/v5 v5.0.10 h1:rLz5avzKpjqxrYwXNfmjkrYYXOyLJd37pz53UFHC6vk=
github.com/go-chi/chi/v5 v5.0.10/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/go-chi/cors v1.2.1 h1:xEC8UT3Rlp2QuWNEr4Fs/c2EAGVKBwy/1vHx3bppil4=
github.com/go-chi/cors v1.2.1/go.mod h1:sSbTewc+6wYHBBCW7ytsFSn836hqM7JxpglAy2Vzc58=
github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxIA=
github.com/go-errors/errors v1.4.2/go.mod h1:sIVyrIiJhuEF+Pj9Ebtd6P/rEYROXFi3BopGUQ5a5Og=
github.com/go-jose/go-jose/v4 v4.0.1 h1:QVEPDE3OluqXBQZDcnNvQrInro2h0e4eqNbnZSWqS6U=
github.com/go-jose/go-jose/v4 v4.0.1/go.mod h1:WVf9LFMHh/QVrmqrOfqun0C45tMe3RoiKJMPvgWwLfY=
github.com/go-jose/go-jose/v4 v4.1.0 h1:cYSYxd3pw5zd2FSXk2vGdn9igQU2PS8MuxrCOCl0FdY=
github.com/go-jose/go-jose/v4 v4.1.0/go.mod h1:GG/vqmYm3Von2nYiB2vGTXzdoNKE5tix5tuc6iAd+sw=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.1 h1:pKouT5E8xu9zeFC39JXRDukb6JFQPXM5p5I91188VAQ=
github.com/go-logr/logr v1.4.1/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
github.com/go-playground/locales v0.14.0 h1:u50s323jtVGugKlcYeyzC0etD1HifMjqmJqb8WugfUU=
github.com/go-playground/locales v0.14.0/go.mod h1:sawfccIbzZTqEDETgFXqTho0QybSa7l++s0DH+LDiLs=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
github.com/go-playground/universal-translator v0.18.0 h1:82dyy6p4OuJq4/CByFNOn/jYrnRPArHwAcmLoJZxyho=
github.com/go-playground/universal-translator v0.18.0/go.mod h1:UvRDBj+xPUEGrFYl+lu/H90nyDXpg0fqeB/AQUGNTVA=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.2.0/go.mod h1:uOYAAleCW8F/7oMFd6aG0GOhaH6EGOAJShg8Id5JGkI=
github.com/go-playground/validator/v10 v10.11.1 h1:prmOlTVv+YjZjmRmNSF3VmspqJIxJWXmqUsHwfTRRkQ=
github.com/go-playground/validator/v10 v10.11.1/go.mod h1:i+3WkQ1FvaUjjxh1kSvIA4dMGDBiPU55YFDl0WbKdWU=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 h1:tfuBGBXKqDEevZMzYi5KSi8KkcZtzBcTgAUUtapy0OI=
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572/go.mod h1:9Pwr4B2jHnOSGXyyzV8ROjYa2ojvAY6HCGYYfMoC3Ls=
github.com/go-playground/validator/v10 v10.15.1 h1:BSe8uhN+xQ4r5guV/ywQI4gO59C2raYcGffYWZEjZzM=
github.com/go-playground/validator/v10 v10.15.1/go.mod h1:9iXMNT7sEkjXb0I+enO7QXmzG6QCsPWY4zveKFVRSyU=
github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI=
github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8=
github.com/gobwas/httphead v0.0.0-20180130184737-2c6c146eadee/go.mod h1:L0fX3K22YWvt/FAX9NnzrNzcI4wNYi9Yku4O0LKYflo=
github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU=
github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
@@ -83,34 +92,31 @@ github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6Wezm
github.com/gobwas/ws v1.0.2/go.mod h1:szmBTxLgaFppYjEmNtny/v3w89xOydFnnZMcgRRu/EM=
github.com/gobwas/ws v1.2.1 h1:F2aeBZrm2NDsc7vbovKrWSogd4wvfAxg0FQ89/iqOTk=
github.com/gobwas/ws v1.2.1/go.mod h1:hRKAFb8wOxFROYNsT1bqfWnhX+b5MFeJM9r2ZSwg/KY=
github.com/goccy/go-json v0.9.11 h1:/pAaQDLHEoCq/5FFmSKBswWmK6H0e8g4159Kc/X/nqk=
github.com/goccy/go-json v0.9.11/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.3.5/go.mod h1:6O5/vntMXwX2lRkT1hjjk0nAC1IDOTvTlVgjlRvqsdk=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8=
github.com/google/gopacket v1.1.19/go.mod h1:iJ8V8n6KS+z2U1A8pUwu8bW5SyEMkXJB8Yo/Vo+TKTo=
github.com/google/pprof v0.0.0-20230817174616-7a8ec2ada47b h1:h9U78+dx9a4BKdQkBBos92HalKpaGKHrp+3Uo6yTodo=
github.com/google/pprof v0.0.0-20230817174616-7a8ec2ada47b/go.mod h1:czg5+yv1E0ZGTi6S6vVK1mke0fV+FaUhNGcd6VRS9Ik=
github.com/google/pprof v0.0.0-20250418163039-24c5476c6587 h1:b/8HpQhvKLSNzH5oTXN2WkNcMl6YB5K3FRbb+i+Ml34=
github.com/google/pprof v0.0.0-20250418163039-24c5476c6587/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.4.1/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1 h1:/c3QmbOGMGTOumP2iT/rCwB7b0QDGLKzqOmktBjT+Is=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.19.1/go.mod h1:5SN9VR2LTsRFsrEC6FHgRbTWrTHu6tqPeKxEQv15giM=
github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc=
github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 h1:5ZPtiqj0JL5oKWmcsq4VMaAW5ukBEgSGXEN89zeH1Jo=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3/go.mod h1:ndYquD05frm2vACXE1nsccT4oJzjhw2arTS2cpUD1PI=
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645 h1:MJG/KsmcqMwFAkh8mTnAwhyKoB+sTAnY4CACC110tbU=
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645/go.mod h1:6iZfnjpejD4L/4DwD7NryNaJyCQdzwWwH2MWhCA90Kw=
github.com/ipostelnik/cli/v2 v2.3.1-0.20210324024421-b6ea8234fe3d h1:PRDnysJ9dF1vUMmEzBu6aHQeUluSQy4eWH3RsSSy/vI=
@@ -119,29 +125,29 @@ github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/u
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/compress v1.10.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.15.11 h1:Lcadnb3RKGin4FYM/orgq0qde+nc15E5Cbqg4B9Sx9c=
github.com/klauspost/compress v1.15.11/go.mod h1:QPwzmACJjUTFsnSHH934V6woptycfrDDJnH7hvFVbGM=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
github.com/klauspost/cpuid/v2 v2.2.5 h1:0E5MSMDEoAulmXNFquVs//DdoomxaoTY1kUhbc/qbZg=
github.com/klauspost/cpuid/v2 v2.2.5/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII=
github.com/leodido/go-urn v1.2.1 h1:BqpAaACuzVSgi/VLzGZIobT2z4v53pjosyNd9Yv6n/w=
github.com/leodido/go-urn v1.2.1/go.mod h1:zt4jvISO2HfUBqxjfIshjdMTYS56ZS/qv49ictyFfxY=
github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q=
github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-isatty v0.0.16 h1:bq3VjFmv/sOjHtdEhmkEV4x1AJtvUvOJ2PFAZ5+peKQ=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
github.com/miekg/dns v1.1.58 h1:ca2Hdkz+cDg/7eNF6V56jjzuZ4aCAE+DbVkILdQWG/4=
github.com/miekg/dns v1.1.58/go.mod h1:Ypv+3b/KadlvW9vJfXOTf300O4UqaHFzFCuHz+rPkBY=
github.com/miekg/dns v1.1.66 h1:FeZXOS3VCVsKnEAd+wBkjMC3D2K+ww66Cq3VnCINuJE=
github.com/miekg/dns v1.1.66/go.mod h1:jGFzBsSNbJw6z1HYut1RKBKHA9PBdxeHrZG8J+gC2WE=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -150,33 +156,38 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJ
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/onsi/ginkgo/v2 v2.13.0 h1:0jY9lJquiL8fcf3M4LAXN5aMlS/b2BV86HFFPCPMgE4=
github.com/onsi/ginkgo/v2 v2.13.0/go.mod h1:TE309ZR8s5FsKKpuB1YAQYBzCaAfUgatB/xlT/ETL/o=
github.com/onsi/gomega v1.27.10 h1:naR28SdDFlqrG6kScpT8VWpu1xWY5nJRCF3XaYyBjhI=
github.com/onsi/gomega v1.27.10/go.mod h1:RsS8tutOdbdgzbPtzzATp12yT7kM5I5aElG3evPbQ0M=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/onsi/ginkgo/v2 v2.23.4 h1:ktYTpKJAVZnDT4VjxSbiBenUjmlL/5QkBEocaWXiQus=
github.com/onsi/ginkgo/v2 v2.23.4/go.mod h1:Bt66ApGPBFzHyR+JO10Zbt0Gsp4uWxu5mIOTusL46e8=
github.com/onsi/gomega v1.36.3 h1:hID7cr8t3Wp26+cYnfcjR6HpJ00fdogN6dqZ1t6IylU=
github.com/onsi/gomega v1.36.3/go.mod h1:8D9+Txp43QWKhM24yyOBEdpkzN8FvJyAwecBgsU4KU0=
github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=
github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=
github.com/pelletier/go-toml/v2 v2.0.5 h1:ipoSadvV8oGUjnUbMub59IDPPwfxF694nG/jwbMiyQg=
github.com/pelletier/go-toml/v2 v2.0.5/go.mod h1:OMHamSCAODeSsVrwwvcJOaoN0LIUIaFVNZzmWyNfXas=
github.com/philhofer/fwd v1.1.2 h1:bnDivRJ1EWPjUIRXV5KfORO897HTbpFAQddBdE8t7Gw=
github.com/philhofer/fwd v1.1.2/go.mod h1:qkPdfjR2SIEbspLqpe1tO4n5yICnr2DY7mqEx2tUTP0=
github.com/pelletier/go-toml/v2 v2.0.9 h1:uH2qQXheeefCCkuBBSLi7jCiSmj3VRh2+Goq2N7Xxu0=
github.com/pelletier/go-toml/v2 v2.0.9/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc=
github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c h1:dAMKvw0MlJT1GshSTtih8C2gDs04w8dReiOGXrGLNoY=
github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c/go.mod h1:RqIHx9QI14HlwKwm98g9Re5prTQ6LdeRQn+gXJFxsJM=
github.com/pingcap/errors v0.11.4 h1:lFuQV/oaUMGcD2tqt+01ROSmJs75VG1ToEOkZIZ4nE4=
github.com/pingcap/errors v0.11.4/go.mod h1:Oi8TUi2kEtXXLMJk9l1cGmz20kV3TaQ0usTwv5KuLY8=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v1.19.1 h1:wZWJDwK+NameRJuPGDhlnFgx8e8HN3XHQeLaYJFJBOE=
github.com/prometheus/client_golang v1.19.1/go.mod h1:mP78NwGzrVks5S2H6ab8+ZZGJLZUq1hoULYBAYBw1Ho=
github.com/prometheus/client_model v0.6.0 h1:k1v3CzpSRUTrKMppY35TLwPvxHqBu0bYgxZzqGIgaos=
github.com/prometheus/client_model v0.6.0/go.mod h1:NTQHnmxFpouOD0DpvP4XujX3CdOAGQPoaGhyTchlyt8=
github.com/prometheus/common v0.53.0 h1:U2pL9w9nmJwJDa4qqLQ3ZaePJ6ZTwt7cMD3AG3+aLCE=
github.com/prometheus/common v0.53.0/go.mod h1:BrxBKv3FWBIGXw89Mg1AeBq7FSyRzXWI3l3e7W3RN5U=
github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo=
github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prashantv/gostub v1.1.0 h1:BTyx3RfQjRHnUWaGF9oQos79AlQ5k8WNktv7VGvVH4g=
github.com/prashantv/gostub v1.1.0/go.mod h1:A5zLQHz7ieHGG7is6LLXLz7I8+3LZzsrV0P1IAHhP5U=
github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/Y92Vm0Zc6Q=
github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.64.0 h1:pdZeA+g617P7oGv1CzdTzyeShxAGrTBsolKNOLQPGO4=
github.com/prometheus/common v0.64.0/go.mod h1:0gZns+BLRQ3V6NdaerOhMbwwRbNh9hkGINtQAsP5GS8=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
github.com/rs/xid v1.2.1/go.mod h1:+uKXf+4Djp6Md1KODXJxgGQPKngRmWyn10oCKFzNHOQ=
github.com/rs/zerolog v1.20.0 h1:38k9hgtUBdxFwE34yS8rTHmHBa4eN16E4DJlv177LNs=
github.com/rs/zerolog v1.20.0/go.mod h1:IzD0RJ65iWH0w97OQQebJEvTZYvsCUm9WVLWBQrJRjo=
@@ -185,115 +196,104 @@ github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tinylib/msgp v1.1.8 h1:FCXC1xanKO4I8plpHGH2P7koL/RzZs12l/+r7vakfm0=
github.com/tinylib/msgp v1.1.8/go.mod h1:qkpG+2ldGg4xRFmx+jfTvZPxfGFhi64BcnL9vkCm/Tw=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tinylib/msgp v1.2.5 h1:WeQg1whrXRFiZusidTQqzETkRpGjFjcIhW6uqWH09po=
github.com/tinylib/msgp v1.2.5/go.mod h1:ykjzy2wzgrlvpDCRc4LA8UXy6D8bzMSuAF3WD57Gok0=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go v1.1.7 h1:/68gy2h+1mWMrwZFeD1kQialdSzAb432dtpeJ42ovdo=
github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw=
github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY=
github.com/ugorji/go/codec v1.2.7 h1:YPXUKf7fYbp/y8xloBqZOw2qaVggbfwMlI8WM3wZUJ0=
github.com/ugorji/go/codec v1.2.7/go.mod h1:WGN1fab3R1fzQlVQTkfxVtIBhWDRqOviHU95kRgeqEY=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/propagators v0.22.0 h1:KGdv58M2//veiYLIhb31mofaI2LgkIPXXAZVeYVyfd8=
go.opentelemetry.io/contrib/propagators v0.22.0/go.mod h1:xGOuXr6lLIF9BXipA4pm6UuOSI0M98U6tsI3khbOiwU=
go.opentelemetry.io/otel v1.0.0-RC2/go.mod h1:w1thVQ7qbAy8MHb0IFj8a5Q2QU0l2ksf8u/CN8m3NOM=
go.opentelemetry.io/otel v1.26.0 h1:LQwgL5s/1W7YiiRwxf03QGnWLb2HW4pLiAhaA5cZXBs=
go.opentelemetry.io/otel v1.26.0/go.mod h1:UmLkJHUAidDval2EICqBMbnAd0/m2vmpf/dAM+fvFs4=
go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ=
go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.26.0 h1:1u/AyyOqAWzy+SkPxDpahCNZParHV8Vid1RnI2clyDE=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.26.0/go.mod h1:z46paqbJ9l7c9fIPCXTqTGwhQZ5XoTIsfeFYWboizjs=
go.opentelemetry.io/otel/metric v1.26.0 h1:7S39CLuY5Jgg9CrnA9HHiEjGMF/X2VHvoXGgSllRz30=
go.opentelemetry.io/otel/metric v1.26.0/go.mod h1:SY+rHOI4cEawI9a7N1A4nIg/nTQXe1ccCNWYOJUrpX4=
go.opentelemetry.io/otel/sdk v1.26.0 h1:Y7bumHf5tAiDlRYFmGqetNcLaVUZmh4iYfmGxtmz7F8=
go.opentelemetry.io/otel/sdk v1.26.0/go.mod h1:0p8MXpqLeJ0pzcszQQN4F0S5FVjBLgypeGSngLsmirs=
go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M=
go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE=
go.opentelemetry.io/otel/sdk v1.35.0 h1:iPctf8iprVySXSKJffSS79eOjl9pvxV9ZqOWT0QejKY=
go.opentelemetry.io/otel/sdk v1.35.0/go.mod h1:+ga1bZliga3DxJ3CQGg3updiaAJoNECOgJREo9KHGQg=
go.opentelemetry.io/otel/sdk/metric v1.35.0 h1:1RriWBmCKgkeHEhM7a2uMjMUfP7MsOF5JpUCaEqEI9o=
go.opentelemetry.io/otel/sdk/metric v1.35.0/go.mod h1:is6XYCUMpcKi+ZsOvfluY5YstFnhW0BidkR+gL+qN+w=
go.opentelemetry.io/otel/trace v1.0.0-RC2/go.mod h1:JPQ+z6nNw9mqEGT8o3eoPTdnNI+Aj5JcxEsVGREIAy4=
go.opentelemetry.io/otel/trace v1.26.0 h1:1ieeAUb4y0TE26jUFrCIXKpTuVK7uJGN9/Z/2LP5sQA=
go.opentelemetry.io/otel/trace v1.26.0/go.mod h1:4iDxvGDQuUkHve82hJJ8UqrwswHYsZuWCBllGV2U2y0=
go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs=
go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc=
go.opentelemetry.io/proto/otlp v1.2.0 h1:pVeZGk7nXDC9O2hncA6nHldxEjm6LByfA2aN8IOkz94=
go.opentelemetry.io/proto/otlp v1.2.0/go.mod h1:gGpR8txAl5M03pDhMC79G6SdqNV26naRm/KDsgaHD8A=
go.uber.org/automaxprocs v1.4.0 h1:CpDZl6aOlLhReez+8S3eEotD7Jx0Os++lemPlMULQP0=
go.uber.org/automaxprocs v1.4.0/go.mod h1:/mTEdr7LvHhs0v7mjdxDreTz1OG5zdZGqgOnhWiR/+Q=
go.uber.org/mock v0.5.0 h1:KAMbZvZPyBPWgD14IrIQ38QCyjwpvVVV6K/bHl1IwQU=
go.uber.org/mock v0.5.0/go.mod h1:ge71pBPLYDk7QIi1LupWxdAykm7KIEFchiOqd6z7qMM=
go.uber.org/automaxprocs v1.6.0 h1:O3y2/QNTOdbF+e/dpXNNW7Rx2hZ4sTIPyybbxyNqTUs=
go.uber.org/automaxprocs v1.6.0/go.mod h1:ifeIMSnPZuznNm6jmdzmU3/bfk01Fe2fotchwEFJ8r8=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/mock v0.5.1 h1:ASgazW/qBmR+A32MYFDB6E2POoTgOwT509VP0CT/fjs=
go.uber.org/mock v0.5.1/go.mod h1:ge71pBPLYDk7QIi1LupWxdAykm7KIEFchiOqd6z7qMM=
golang.org/x/arch v0.4.0 h1:A8WCeEWhLwPBKNbFi5Wv5UTCBx5zzubnXDlMOFAzFMc=
golang.org/x/arch v0.4.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842 h1:vr/HnozRka3pE4EsMEg1lgkXJkTFJCVUX+S/ZT6wYzM=
golang.org/x/exp v0.0.0-20240506185415-9bf2ced13842/go.mod h1:XtvwrStGgqGPLc4cjQfWqZHG1YFdYs6swckp8vpsjnc=
golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8=
golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.18.0 h1:5+9lSbEzPSdWkH32vYPBwEpX8KwDbM52Ud9xBUvNlb0=
golang.org/x/mod v0.18.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.24.0 h1:ZfthKaKaT4NrhGVZHO1/WDTwGES4De8KtWO0SIbNJMU=
golang.org/x/mod v0.24.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.26.0 h1:soB7SVo0PWrY4vPW/+ay0jKDNScG2X9wFeYlXIvJsOQ=
golang.org/x/net v0.26.0/go.mod h1:5YKkiSynbBIh3p6iOc/vibscux0x38BZDkn8sCUPxHE=
golang.org/x/oauth2 v0.18.0 h1:09qnuIAgzdx1XplqJvW6CQqMCtGZykZWcXzPMPUusvI=
golang.org/x/oauth2 v0.18.0/go.mod h1:Wf7knwG0MPoWIMMBgFlEaSUDaKskp0dCfrlJRJXbBi8=
golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY=
golang.org/x/net v0.40.0/go.mod h1:y0hY0exeL2Pku80/zKK7tpntoX23cqL3Oa6njdgRtds=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ=
golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.27.0 h1:WP60Sv1nlK1T6SupCHbXzSaN0b9wUmsPoRS9b61A23Q=
golang.org/x/term v0.27.0/go.mod h1:iMsnZpn0cago0GOrHO2+Y7u7JPn5AylBrcoWkElMTSM=
golang.org/x/sys v0.33.0 h1:q3i8TbbEz+JRD9ywIRlyRAQbM0qF7hu24q3teo2hbuw=
golang.org/x/sys v0.33.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/term v0.32.0 h1:DR4lr0TjUs3epypdhTOkMmuF5CDFJ/8pOnbzMZPQ7bg=
golang.org/x/term v0.32.0/go.mod h1:uZG1FhGx848Sqfsq4/DlJr3xGGsYMu/L5GW4abiaEPQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190828213141-aed303cbaa74/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.22.0 h1:gqSGLZqv+AI9lIQzniJ0nZDRG5GBPsSi+DRNHWNz6yA=
golang.org/x/tools v0.22.0/go.mod h1:aCwcsjqvq7Yqt6TNyX7QMU2enbQ/Gt0bo6krSeEri+c=
golang.org/x/tools v0.32.0 h1:Q7N1vhpkQv7ybVzLFtTjvQya2ewbwNDZzUgfXGqtMWU=
golang.org/x/tools v0.32.0/go.mod h1:ZxrU41P/wAbZD8EDa6dDCa6XfpkhJ7HFMjHJXfBDu8s=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.6.8 h1:IhEN5q69dyKagZPYMSdIjS2HqprW324FRQZJcGqPAsM=
google.golang.org/appengine v1.6.8/go.mod h1:1jJ3jBArFh5pcgW8gCtRJnepW8FzD1V44FJffLiz/Ds=
google.golang.org/genproto/googleapis/api v0.0.0-20240311132316-a219d84964c2 h1:rIo7ocm2roD9DcFIX67Ym8icoGCKSARAiPljFhh5suQ=
google.golang.org/genproto/googleapis/api v0.0.0-20240311132316-a219d84964c2/go.mod h1:O1cOfN1Cy6QEYr7VxtjOyP5AdAuR0aJ/MYZaaof623Y=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237 h1:NnYq6UN9ReLM9/Y01KWNOWyI5xQ9kbIms5GGJVwS/Yc=
google.golang.org/genproto/googleapis/rpc v0.0.0-20240318140521-94a12d6c2237/go.mod h1:WtryC6hu0hhx87FDGxWCDptyssuo68sk10vYjF+T9fY=
google.golang.org/grpc v1.63.2 h1:MUeiw1B2maTVZthpU5xvASfTh3LDbxHd6IJ6QQVU+xM=
google.golang.org/grpc v1.63.2/go.mod h1:WAX/8DgncnokcFUldAxq7GeB5DXHDbMF+lLvDomNkRA=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.34.1 h1:9ddQBjfCyZPOHPUiPxpYESBLc+T8P3E+Vo4IbKZgFWg=
google.golang.org/protobuf v1.34.1/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
google.golang.org/genproto/googleapis/api v0.0.0-20250505200425-f936aa4a68b2 h1:vPV0tzlsK6EzEDHNNH5sa7Hs9bd7iXR7B1tSiPepkV0=
google.golang.org/genproto/googleapis/api v0.0.0-20250505200425-f936aa4a68b2/go.mod h1:pKLAc5OolXC3ViWGI62vvC0n10CpwAtRcTNCFwTKBEw=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250512202823-5a2f75b736a9 h1:IkAfh6J/yllPtpYFU0zZN1hUPYdT0ogkBT/9hMxHjvg=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250512202823-5a2f75b736a9/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
google.golang.org/grpc v1.72.2 h1:TdbGzwb82ty4OusHWepvFWGLgIbNo1/SUynEN0ssqv8=
google.golang.org/grpc v1.72.2/go.mod h1:wH5Aktxcg25y1I3w7H69nHfXdOG3UiadoBtjh3izSDM=
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/natefinch/lumberjack.v2 v2.0.0 h1:1Lc07Kr7qY4U2YPouBjpCLxpiyxIVoxqXgkXLknAOE8=
+1 -1
View File
@@ -1,4 +1,4 @@
//go:build windows
//go:build windows && cgo
package ingress
+4 -4
View File
@@ -113,7 +113,7 @@ func ParseIngressFromConfigAndCLI(conf *config.Configuration, c *cli.Context, lo
// If no token is provided, the probability of NOT being a remotely managed tunnel is higher.
// So, we should warn the user that no ingress rules were found, because remote configuration will most likely not exist.
if !c.IsSet("token") {
log.Warn().Msgf(ErrNoIngressRulesCLI.Error())
log.Warn().Msg(ErrNoIngressRulesCLI.Error())
}
return newDefaultOrigin(c, log), nil
}
@@ -378,17 +378,17 @@ func validateHostname(r config.UnvalidatedIngressRule, ruleIndex, totalRules int
}
// ONLY the last rule should catch all hostnames.
if !isLastRule && isCatchAllRule {
return errRuleShouldNotBeCatchAll{index: ruleIndex, hostname: r.Hostname}
return ruleShouldNotBeCatchAllError{index: ruleIndex, hostname: r.Hostname}
}
return nil
}
type errRuleShouldNotBeCatchAll struct {
type ruleShouldNotBeCatchAllError struct {
index int
hostname string
}
func (e errRuleShouldNotBeCatchAll) Error() string {
func (e ruleShouldNotBeCatchAllError) Error() string {
return fmt.Sprintf("Rule #%d is matching the hostname '%s', but "+
"this will match every hostname, meaning the rules which follow it "+
"will never be triggered.", e.index+1, e.hostname)
+6 -14
View File
@@ -19,7 +19,7 @@ import (
type OriginConnection interface {
// Stream should generally be implemented as a bidirectional io.Copy.
Stream(ctx context.Context, tunnelConn io.ReadWriter, log *zerolog.Logger)
Close()
Close() error
}
type streamHandlerFunc func(originConn io.ReadWriter, remoteConn net.Conn, log *zerolog.Logger)
@@ -48,16 +48,7 @@ func (tc *tcpConnection) Write(b []byte) (int, error) {
}
}
nBytes, err := tc.Conn.Write(b)
if err != nil {
tc.logger.Err(err).Msg("Error writing to the TCP connection")
}
return nBytes, err
}
func (tc *tcpConnection) Close() {
tc.Conn.Close()
return tc.Conn.Write(b)
}
// tcpOverWSConnection is an OriginConnection that streams to TCP over WS.
@@ -75,8 +66,8 @@ func (wc *tcpOverWSConnection) Stream(ctx context.Context, tunnelConn io.ReadWri
wsConn.Close()
}
func (wc *tcpOverWSConnection) Close() {
wc.conn.Close()
func (wc *tcpOverWSConnection) Close() error {
return wc.conn.Close()
}
// socksProxyOverWSConnection is an OriginConnection that streams SOCKS connections over WS.
@@ -95,5 +86,6 @@ func (sp *socksProxyOverWSConnection) Stream(ctx context.Context, tunnelConn io.
wsConn.Close()
}
func (sp *socksProxyOverWSConnection) Close() {
func (sp *socksProxyOverWSConnection) Close() error {
return nil
}
+146
View File
@@ -0,0 +1,146 @@
package ingress
import (
"context"
"fmt"
"net"
"net/netip"
"sync"
"time"
"github.com/rs/zerolog"
)
// OriginTCPDialer provides a TCP dial operation to a requested address.
type OriginTCPDialer interface {
DialTCP(ctx context.Context, addr netip.AddrPort) (net.Conn, error)
}
// OriginUDPDialer provides a UDP dial operation to a requested address.
type OriginUDPDialer interface {
DialUDP(addr netip.AddrPort) (net.Conn, error)
}
// OriginDialer provides both TCP and UDP dial operations to an address.
type OriginDialer interface {
OriginTCPDialer
OriginUDPDialer
}
type OriginConfig struct {
// The default Dialer used if no reserved services are found for an origin request.
DefaultDialer OriginDialer
// Timeout on write operations for TCP connections to the origin.
TCPWriteTimeout time.Duration
}
// OriginDialerService provides a proxy TCP and UDP dialer to origin services while allowing reserved
// services to be provided. These reserved services are assigned to specific [netip.AddrPort]s
// and provide their own [OriginDialer]'s to handle origin dialing per protocol.
type OriginDialerService struct {
// Reserved TCP services for reserved AddrPort values
reservedTCPServices map[netip.AddrPort]OriginTCPDialer
// Reserved UDP services for reserved AddrPort values
reservedUDPServices map[netip.AddrPort]OriginUDPDialer
// The default Dialer used if no reserved services are found for an origin request
defaultDialer OriginDialer
defaultDialerM sync.RWMutex
// Write timeout for TCP connections
writeTimeout time.Duration
logger *zerolog.Logger
}
func NewOriginDialer(config OriginConfig, logger *zerolog.Logger) *OriginDialerService {
return &OriginDialerService{
reservedTCPServices: map[netip.AddrPort]OriginTCPDialer{},
reservedUDPServices: map[netip.AddrPort]OriginUDPDialer{},
defaultDialer: config.DefaultDialer,
writeTimeout: config.TCPWriteTimeout,
logger: logger,
}
}
// AddReservedService adds a reserved virtual service to dial to.
// Not locked and expected to be initialized before calling first dial and not afterwards.
func (d *OriginDialerService) AddReservedService(service OriginDialer, addrs []netip.AddrPort) {
for _, addr := range addrs {
d.reservedTCPServices[addr] = service
d.reservedUDPServices[addr] = service
}
}
// UpdateDefaultDialer updates the default dialer.
func (d *OriginDialerService) UpdateDefaultDialer(dialer *Dialer) {
d.defaultDialerM.Lock()
defer d.defaultDialerM.Unlock()
d.defaultDialer = dialer
}
// DialTCP will perform a dial TCP to the requested addr.
func (d *OriginDialerService) DialTCP(ctx context.Context, addr netip.AddrPort) (net.Conn, error) {
conn, err := d.dialTCP(ctx, addr)
if err != nil {
return nil, err
}
// Assign the write timeout for the TCP operations
return &tcpConnection{
Conn: conn,
writeTimeout: d.writeTimeout,
logger: d.logger,
}, nil
}
func (d *OriginDialerService) dialTCP(ctx context.Context, addr netip.AddrPort) (net.Conn, error) {
// Check to see if any reserved services are available for this addr and call their dialer instead.
if dialer, ok := d.reservedTCPServices[addr]; ok {
return dialer.DialTCP(ctx, addr)
}
d.defaultDialerM.RLock()
dialer := d.defaultDialer
d.defaultDialerM.RUnlock()
return dialer.DialTCP(ctx, addr)
}
// DialUDP will perform a dial UDP to the requested addr.
func (d *OriginDialerService) DialUDP(addr netip.AddrPort) (net.Conn, error) {
// Check to see if any reserved services are available for this addr and call their dialer instead.
if dialer, ok := d.reservedUDPServices[addr]; ok {
return dialer.DialUDP(addr)
}
d.defaultDialerM.RLock()
dialer := d.defaultDialer
d.defaultDialerM.RUnlock()
return dialer.DialUDP(addr)
}
type Dialer struct {
Dialer net.Dialer
}
func NewDialer(config WarpRoutingConfig) *Dialer {
return &Dialer{
Dialer: net.Dialer{
Timeout: config.ConnectTimeout.Duration,
KeepAlive: config.TCPKeepAlive.Duration,
},
}
}
func (d *Dialer) DialTCP(ctx context.Context, dest netip.AddrPort) (net.Conn, error) {
conn, err := d.Dialer.DialContext(ctx, "tcp", dest.String())
if err != nil {
return nil, fmt.Errorf("unable to dial tcp to origin %s: %w", dest, err)
}
return conn, nil
}
func (d *Dialer) DialUDP(dest netip.AddrPort) (net.Conn, error) {
conn, err := d.Dialer.Dial("udp", dest.String())
if err != nil {
return nil, fmt.Errorf("unable to dial udp to origin %s: %w", dest, err)
}
return conn, nil
}
+2 -3
View File
@@ -66,7 +66,7 @@ func (o *httpService) SetOriginServerName(req *http.Request) {
}
return tls.Client(conn, &tls.Config{
RootCAs: o.transport.TLSClientConfig.RootCAs,
InsecureSkipVerify: o.transport.TLSClientConfig.InsecureSkipVerify,
InsecureSkipVerify: o.transport.TLSClientConfig.InsecureSkipVerify, // nolint: gosec
ServerName: req.Host,
}), nil
}
@@ -74,7 +74,7 @@ func (o *httpService) SetOriginServerName(req *http.Request) {
func (o *statusCode) RoundTrip(_ *http.Request) (*http.Response, error) {
if o.defaultResp {
o.log.Warn().Msgf(ErrNoIngressRulesCLI.Error())
o.log.Warn().Msg(ErrNoIngressRulesCLI.Error())
}
resp := &http.Response{
StatusCode: o.code,
@@ -114,7 +114,6 @@ func (o *tcpOverWSService) EstablishConnection(ctx context.Context, dest string,
streamHandler: o.streamHandler,
}
return originConn, nil
}
func (o *socksProxyOverWSService) EstablishConnection(_ context.Context, _ string, _ *zerolog.Logger) (OriginConnection, error) {
-46
View File
@@ -1,46 +0,0 @@
package ingress
import (
"fmt"
"io"
"net"
"net/netip"
)
type UDPProxy interface {
io.ReadWriteCloser
LocalAddr() net.Addr
}
type udpProxy struct {
*net.UDPConn
}
func DialUDP(dstIP net.IP, dstPort uint16) (UDPProxy, error) {
dstAddr := &net.UDPAddr{
IP: dstIP,
Port: int(dstPort),
}
// We use nil as local addr to force runtime to find the best suitable local address IP given the destination
// address as context.
udpConn, err := net.DialUDP("udp", nil, dstAddr)
if err != nil {
return nil, fmt.Errorf("unable to create UDP proxy to origin (%v:%v): %w", dstIP, dstPort, err)
}
return &udpProxy{udpConn}, nil
}
func DialUDPAddrPort(dest netip.AddrPort) (*net.UDPConn, error) {
addr := net.UDPAddrFromAddrPort(dest)
// We use nil as local addr to force runtime to find the best suitable local address IP given the destination
// address as context.
udpConn, err := net.DialUDP("udp", nil, addr)
if err != nil {
return nil, fmt.Errorf("unable to dial udp to origin %s: %w", dest, err)
}
return udpConn, nil
}
+219
View File
@@ -0,0 +1,219 @@
package origins
import (
"context"
"crypto/rand"
"math/big"
"net"
"net/netip"
"slices"
"sync"
"time"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/ingress"
)
const (
// We need a DNS record:
// 1. That will be around for as long as cloudflared is
// 2. That Cloudflare controls: to allow us to make changes if needed
// 3. That is an external record to a typical customer's network: enforcing that the DNS request go to the
// local DNS resolver over any local /etc/host configurations setup.
// 4. That cloudflared would normally query: ensuring that users with a positive security model for DNS queries
// don't need to adjust anything.
//
// This hostname is one that used during the edge discovery process and as such satisfies the above constraints.
defaultLookupHost = "region1.v2.argotunnel.com"
defaultResolverPort uint16 = 53
// We want the refresh time to be short to accommodate DNS resolver changes locally, but not too frequent as to
// shuffle the resolver if multiple are configured.
refreshFreq = 5 * time.Minute
refreshTimeout = 5 * time.Second
)
var (
// Virtual DNS service address
VirtualDNSServiceAddr = netip.AddrPortFrom(netip.MustParseAddr("2606:4700:0cf1:2000:0000:0000:0000:0001"), 53)
defaultResolverAddr = netip.AddrPortFrom(netip.MustParseAddr("127.0.0.1"), defaultResolverPort)
)
type netDial func(network string, address string) (net.Conn, error)
// DNSResolverService will make DNS requests to the local DNS resolver via the Dial method.
type DNSResolverService struct {
addresses []netip.AddrPort
addressesM sync.RWMutex
static bool
dialer ingress.OriginDialer
resolver peekResolver
logger *zerolog.Logger
metrics Metrics
}
func NewDNSResolverService(dialer ingress.OriginDialer, logger *zerolog.Logger, metrics Metrics) *DNSResolverService {
return &DNSResolverService{
addresses: []netip.AddrPort{defaultResolverAddr},
dialer: dialer,
resolver: &resolver{dialFunc: net.Dial},
logger: logger,
metrics: metrics,
}
}
func NewStaticDNSResolverService(resolverAddrs []netip.AddrPort, dialer ingress.OriginDialer, logger *zerolog.Logger, metrics Metrics) *DNSResolverService {
s := NewDNSResolverService(dialer, logger, metrics)
s.addresses = resolverAddrs
s.static = true
return s
}
func (s *DNSResolverService) DialTCP(ctx context.Context, _ netip.AddrPort) (net.Conn, error) {
s.metrics.IncrementDNSTCPRequests()
dest := s.getAddress()
// The dialer ignores the provided address because the request will instead go to the local DNS resolver.
return s.dialer.DialTCP(ctx, dest)
}
func (s *DNSResolverService) DialUDP(_ netip.AddrPort) (net.Conn, error) {
s.metrics.IncrementDNSUDPRequests()
dest := s.getAddress()
// The dialer ignores the provided address because the request will instead go to the local DNS resolver.
return s.dialer.DialUDP(dest)
}
// StartRefreshLoop is a routine that is expected to run in the background to update the DNS local resolver if
// adjusted while the cloudflared process is running.
// Does not run when the resolver was provided with external resolver addresses via CLI.
func (s *DNSResolverService) StartRefreshLoop(ctx context.Context) {
if s.static {
s.logger.Debug().Msgf("Canceled DNS local resolver refresh loop because static resolver addresses were provided: %s", s.addresses)
return
}
// Call update first to load an address before handling traffic
err := s.update(ctx)
if err != nil {
s.logger.Err(err).Msg("Failed to initialize DNS local resolver")
}
for {
select {
case <-ctx.Done():
return
case <-time.Tick(refreshFreq):
err := s.update(ctx)
if err != nil {
s.logger.Err(err).Msg("Failed to refresh DNS local resolver")
}
}
}
}
func (s *DNSResolverService) update(ctx context.Context) error {
ctx, cancel := context.WithTimeout(ctx, refreshTimeout)
defer cancel()
// Make a standard DNS request to a well-known DNS record that will last a long time
_, err := s.resolver.lookupNetIP(ctx, defaultLookupHost)
if err != nil {
return err
}
// Validate the address before updating internal reference
_, address := s.resolver.addr()
peekAddrPort, err := netip.ParseAddrPort(address)
if err == nil {
s.setAddress(peekAddrPort)
return nil
}
// It's possible that the address didn't have an attached port, attempt to parse just the address and use
// the default port 53
peekAddr, err := netip.ParseAddr(address)
if err != nil {
return err
}
s.setAddress(netip.AddrPortFrom(peekAddr, defaultResolverPort))
return nil
}
// returns the address from the peekResolver or from the static addresses if provided.
// If multiple addresses are provided in the static addresses pick one randomly.
func (s *DNSResolverService) getAddress() netip.AddrPort {
s.addressesM.RLock()
defer s.addressesM.RUnlock()
l := len(s.addresses)
if l <= 0 {
return defaultResolverAddr
}
if l == 1 {
return s.addresses[0]
}
// Only initialize the random selection if there is more than one element in the list.
var i int64 = 0
r, err := rand.Int(rand.Reader, big.NewInt(int64(l)))
// We ignore errors from crypto rand and use index 0; this should be extremely unlikely and the
// list index doesn't need to be cryptographically secure, but linters insist.
if err == nil {
i = r.Int64()
}
return s.addresses[i]
}
// lock and update the address used for the local DNS resolver
func (s *DNSResolverService) setAddress(addr netip.AddrPort) {
s.addressesM.Lock()
defer s.addressesM.Unlock()
if !slices.Contains(s.addresses, addr) {
s.logger.Debug().Msgf("Updating DNS local resolver: %s", addr)
}
// We only store one address when reading the peekResolver, so we just replace the whole list.
s.addresses = []netip.AddrPort{addr}
}
type peekResolver interface {
addr() (network string, address string)
lookupNetIP(ctx context.Context, host string) ([]netip.Addr, error)
}
// resolver is a shim that inspects the go runtime's DNS resolution process to capture the DNS resolver
// address used to complete a DNS request.
type resolver struct {
network string
address string
dialFunc netDial
}
func (r *resolver) addr() (network string, address string) {
return r.network, r.address
}
func (r *resolver) lookupNetIP(ctx context.Context, host string) ([]netip.Addr, error) {
resolver := &net.Resolver{
PreferGo: true,
// Use the peekDial to inspect the results of the DNS resolver used during the LookupIPAddr call.
Dial: r.peekDial,
}
return resolver.LookupNetIP(ctx, "ip", host)
}
func (r *resolver) peekDial(ctx context.Context, network, address string) (net.Conn, error) {
r.network = network
r.address = address
return r.dialFunc(network, address)
}
// NewDNSDialer creates a custom dialer for the DNS resolver service to utilize.
func NewDNSDialer() *ingress.Dialer {
return &ingress.Dialer{
Dialer: net.Dialer{
// We want short timeouts for the DNS requests
Timeout: 5 * time.Second,
// We do not want keep alive since the edge will not reuse TCP connections per request
KeepAlive: -1,
KeepAliveConfig: net.KeepAliveConfig{
Enable: false,
},
},
}
}
+195
View File
@@ -0,0 +1,195 @@
package origins
import (
"context"
"errors"
"net"
"net/netip"
"slices"
"testing"
"time"
"github.com/rs/zerolog"
)
func TestDNSResolver_DefaultResolver(t *testing.T) {
log := zerolog.Nop()
service := NewDNSResolverService(NewDNSDialer(), &log, &noopMetrics{})
mockResolver := &mockPeekResolver{
address: "127.0.0.2:53",
}
service.resolver = mockResolver
validateAddrs(t, []netip.AddrPort{defaultResolverAddr}, service.addresses)
}
func TestStaticDNSResolver_DefaultResolver(t *testing.T) {
log := zerolog.Nop()
addresses := []netip.AddrPort{netip.MustParseAddrPort("1.1.1.1:53"), netip.MustParseAddrPort("1.0.0.1:53")}
service := NewStaticDNSResolverService(addresses, NewDNSDialer(), &log, &noopMetrics{})
mockResolver := &mockPeekResolver{
address: "127.0.0.2:53",
}
service.resolver = mockResolver
validateAddrs(t, addresses, service.addresses)
}
func TestDNSResolver_UpdateResolverAddress(t *testing.T) {
log := zerolog.Nop()
service := NewDNSResolverService(NewDNSDialer(), &log, &noopMetrics{})
mockResolver := &mockPeekResolver{}
service.resolver = mockResolver
tests := []struct {
addr string
expected netip.AddrPort
}{
{"127.0.0.2:53", netip.MustParseAddrPort("127.0.0.2:53")},
// missing port should be added (even though this is unlikely to happen)
{"127.0.0.3", netip.MustParseAddrPort("127.0.0.3:53")},
}
for _, test := range tests {
mockResolver.address = test.addr
// Update the resolver address
err := service.update(t.Context())
if err != nil {
t.Error(err)
}
// Validate expected
validateAddrs(t, []netip.AddrPort{test.expected}, service.addresses)
}
}
func TestStaticDNSResolver_RefreshLoopExits(t *testing.T) {
log := zerolog.Nop()
addresses := []netip.AddrPort{netip.MustParseAddrPort("1.1.1.1:53"), netip.MustParseAddrPort("1.0.0.1:53")}
service := NewStaticDNSResolverService(addresses, NewDNSDialer(), &log, &noopMetrics{})
mockResolver := &mockPeekResolver{
address: "127.0.0.2:53",
}
service.resolver = mockResolver
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
go service.StartRefreshLoop(ctx)
// Wait for the refresh loop to end _and_ not update the addresses
time.Sleep(10 * time.Millisecond)
// Validate expected
validateAddrs(t, addresses, service.addresses)
}
func TestDNSResolver_UpdateResolverAddressInvalid(t *testing.T) {
log := zerolog.Nop()
service := NewDNSResolverService(NewDNSDialer(), &log, &noopMetrics{})
mockResolver := &mockPeekResolver{}
service.resolver = mockResolver
invalidAddresses := []string{
"999.999.999.999",
"localhost",
"255.255.255",
}
for _, addr := range invalidAddresses {
mockResolver.address = addr
// Update the resolver address should not update for these invalid addresses
err := service.update(t.Context())
if err == nil {
t.Error("service update should throw an error")
}
// Validate expected
validateAddrs(t, []netip.AddrPort{defaultResolverAddr}, service.addresses)
}
}
func TestDNSResolver_UpdateResolverErrorIgnored(t *testing.T) {
log := zerolog.Nop()
service := NewDNSResolverService(NewDNSDialer(), &log, &noopMetrics{})
resolverErr := errors.New("test resolver error")
mockResolver := &mockPeekResolver{err: resolverErr}
service.resolver = mockResolver
// Update the resolver address should not update when the resolver cannot complete the lookup
err := service.update(t.Context())
if err != resolverErr {
t.Error("service update should throw an error")
}
// Validate expected
validateAddrs(t, []netip.AddrPort{defaultResolverAddr}, service.addresses)
}
func TestDNSResolver_DialUDPUsesResolvedAddress(t *testing.T) {
log := zerolog.Nop()
mockDialer := &mockDialer{expected: defaultResolverAddr}
service := NewDNSResolverService(mockDialer, &log, &noopMetrics{})
mockResolver := &mockPeekResolver{}
service.resolver = mockResolver
// Attempt a dial to 127.0.0.2:53 which should be ignored and instead resolve to 127.0.0.1:53
_, err := service.DialUDP(netip.MustParseAddrPort("127.0.0.2:53"))
if err != nil {
t.Error(err)
}
}
func TestDNSResolver_DialTCPUsesResolvedAddress(t *testing.T) {
log := zerolog.Nop()
mockDialer := &mockDialer{expected: defaultResolverAddr}
service := NewDNSResolverService(mockDialer, &log, &noopMetrics{})
mockResolver := &mockPeekResolver{}
service.resolver = mockResolver
// Attempt a dial to 127.0.0.2:53 which should be ignored and instead resolve to 127.0.0.1:53
_, err := service.DialTCP(t.Context(), netip.MustParseAddrPort("127.0.0.2:53"))
if err != nil {
t.Error(err)
}
}
type mockPeekResolver struct {
err error
address string
}
func (r *mockPeekResolver) addr() (network, address string) {
return "udp", r.address
}
func (r *mockPeekResolver) lookupNetIP(ctx context.Context, host string) ([]netip.Addr, error) {
// We can return an empty result as it doesn't matter as long as the lookup doesn't fail
return []netip.Addr{}, r.err
}
type mockDialer struct {
expected netip.AddrPort
}
func (d *mockDialer) DialTCP(ctx context.Context, addr netip.AddrPort) (net.Conn, error) {
if d.expected != addr {
return nil, errors.New("unexpected address dialed")
}
return nil, nil
}
func (d *mockDialer) DialUDP(addr netip.AddrPort) (net.Conn, error) {
if d.expected != addr {
return nil, errors.New("unexpected address dialed")
}
return nil, nil
}
func validateAddrs(t *testing.T, expected []netip.AddrPort, actual []netip.AddrPort) {
if len(actual) != len(expected) {
t.Errorf("addresses should only contain one element: %s", actual)
}
for _, e := range expected {
if !slices.Contains(actual, e) {
t.Errorf("missing address: %s in %s", e, actual)
}
}
}
+40
View File
@@ -0,0 +1,40 @@
package origins
import (
"github.com/prometheus/client_golang/prometheus"
)
const (
namespace = "cloudflared"
subsystem = "virtual_origins"
)
type Metrics interface {
IncrementDNSUDPRequests()
IncrementDNSTCPRequests()
}
type metrics struct {
dnsResolverRequests *prometheus.CounterVec
}
func (m *metrics) IncrementDNSUDPRequests() {
m.dnsResolverRequests.WithLabelValues("udp").Inc()
}
func (m *metrics) IncrementDNSTCPRequests() {
m.dnsResolverRequests.WithLabelValues("tcp").Inc()
}
func NewMetrics(registerer prometheus.Registerer) Metrics {
m := &metrics{
dnsResolverRequests: prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: namespace,
Subsystem: subsystem,
Name: "dns_requests_total",
Help: "Total count of DNS requests that have been proxied to the virtual DNS resolver origin",
}, []string{"protocol"}),
}
registerer.MustRegister(m.dnsResolverRequests)
return m
}
+6
View File
@@ -0,0 +1,6 @@
package origins
type noopMetrics struct{}
func (noopMetrics) IncrementDNSUDPRequests() {}
func (noopMetrics) IncrementDNSTCPRequests() {}
+6 -2
View File
@@ -17,6 +17,7 @@ type Config struct {
type ConsoleConfig struct {
noColor bool
asJSON bool
}
type FileConfig struct {
@@ -48,6 +49,7 @@ func createDefaultConfig() Config {
return Config{
ConsoleConfig: &ConsoleConfig{
noColor: false,
asJSON: false,
},
FileConfig: &FileConfig{
Dirname: "",
@@ -67,11 +69,12 @@ func createDefaultConfig() Config {
func CreateConfig(
minLevel string,
disableTerminal bool,
formatJSON bool,
rollingLogPath, nonRollingLogFilePath string,
) *Config {
var console *ConsoleConfig
if !disableTerminal {
console = createConsoleConfig()
console = createConsoleConfig(formatJSON)
}
var file *FileConfig
@@ -95,9 +98,10 @@ func CreateConfig(
}
}
func createConsoleConfig() *ConsoleConfig {
func createConsoleConfig(formatJSON bool) *ConsoleConfig {
return &ConsoleConfig{
noColor: false,
asJSON: formatJSON,
}
}
+37
View File
@@ -0,0 +1,37 @@
package logger
import (
"bytes"
"fmt"
"io"
jsoniter "github.com/json-iterator/go"
)
var json = jsoniter.ConfigCompatibleWithStandardLibrary
// consoleWriter allows us the simplicity to prevent duplicate json keys in the logger events reported.
//
// By default zerolog constructs the json event in parts by appending each additional key after the first. It
// doesn't have any internal state or struct of the json message representation so duplicate keys can be
// inserted without notice and no pruning will occur before writing the log event out to the io.Writer.
//
// To help prevent these duplicate keys, we will decode the json log event and then immediately re-encode it
// again as writing it to the output io.Writer. Since we encode it to a map[string]any, duplicate keys
// are pruned. We pay the cost of decoding and encoding the log event for each time, but helps prevent
// us from needing to worry about adding duplicate keys in the log event from different areas of code.
type consoleWriter struct {
out io.Writer
}
func (c *consoleWriter) Write(p []byte) (n int, err error) {
var evt map[string]any
d := json.NewDecoder(bytes.NewReader(p))
d.UseNumber()
err = d.Decode(&evt)
if err != nil {
return n, fmt.Errorf("cannot decode event: %s", err)
}
e := json.NewEncoder(c.out)
return len(p), e.Encode(evt)
}
+33
View File
@@ -0,0 +1,33 @@
package logger
import (
"bytes"
"strings"
"testing"
"github.com/rs/zerolog"
)
func TestConsoleLoggerDuplicateKeys(t *testing.T) {
r := bytes.NewBuffer(make([]byte, 500))
logger := zerolog.New(&consoleWriter{out: r}).With().Timestamp().Logger()
logger.Debug().Str("test", "1234").Int("number", 45).Str("test", "5678").Msg("log message")
event, err := r.ReadString('\n')
if err != nil {
t.Error(err)
}
if !strings.Contains(event, "\"test\":\"5678\"") {
t.Errorf("log event missing key 'test': %s", event)
}
if !strings.Contains(event, "\"number\":45") {
t.Errorf("log event missing key 'number': %s", event)
}
if !strings.Contains(event, "\"time\":") {
t.Errorf("log event missing key 'time': %s", event)
}
if !strings.Contains(event, "\"level\":\"debug\"") {
t.Errorf("log event missing key 'level': %s", event)
}
}
+14
View File
@@ -149,10 +149,21 @@ func createFromContext(
logLevel := c.String(logLevelFlagName)
logFile := c.String(cfdflags.LogFile)
logDirectory := c.String(logDirectoryFlagName)
var logFormatJSON bool
switch c.String(cfdflags.LogFormatOutput) {
case cfdflags.LogFormatOutputValueJSON:
logFormatJSON = true
case cfdflags.LogFormatOutputValueDefault:
// "default" and unset use the same logger output format
fallthrough
default:
logFormatJSON = false
}
loggerConfig := CreateConfig(
logLevel,
disableTerminal,
logFormatJSON,
logDirectory,
logFile,
)
@@ -177,6 +188,9 @@ func Create(loggerConfig *Config) *zerolog.Logger {
}
func createConsoleLogger(config ConsoleConfig) io.Writer {
if config.asJSON {
return &consoleWriter{out: os.Stderr}
}
consoleOut := os.Stderr
return zerolog.ConsoleWriter{
Out: colorable.NewColorable(consoleOut),
+3 -4
View File
@@ -2,7 +2,6 @@ package orchestration
import (
"encoding/json"
"time"
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/ingress"
@@ -20,9 +19,9 @@ type newLocalConfig struct {
// Config is the original config as read and parsed by cloudflared.
type Config struct {
Ingress *ingress.Ingress
WarpRouting ingress.WarpRoutingConfig
WriteTimeout time.Duration
Ingress *ingress.Ingress
WarpRouting ingress.WarpRoutingConfig
OriginDialerService *ingress.OriginDialerService
// Extra settings used to configure this instance but that are not eligible for remotely management
// ie. (--protocol, --loglevel, ...)
+22 -10
View File
@@ -38,7 +38,9 @@ type Orchestrator struct {
tags []pogs.Tag
// flowLimiter tracks active sessions across the tunnel and limits new sessions if they are above the limit.
flowLimiter cfdflow.Limiter
log *zerolog.Logger
// Origin dialer service to manage egress socket dialing.
originDialerService *ingress.OriginDialerService
log *zerolog.Logger
// orchestrator must not handle any more updates after shutdownC is closed
shutdownC <-chan struct{}
@@ -50,18 +52,20 @@ func NewOrchestrator(ctx context.Context,
config *Config,
tags []pogs.Tag,
internalRules []ingress.Rule,
log *zerolog.Logger) (*Orchestrator, error) {
log *zerolog.Logger,
) (*Orchestrator, error) {
o := &Orchestrator{
// Lowest possible version, any remote configuration will have version higher than this
// Starting at -1 allows a configuration migration (local to remote) to override the current configuration as it
// will start at version 0.
currentVersion: -1,
internalRules: internalRules,
config: config,
tags: tags,
flowLimiter: cfdflow.NewLimiter(config.WarpRouting.MaxActiveFlows),
log: log,
shutdownC: ctx.Done(),
currentVersion: -1,
internalRules: internalRules,
config: config,
tags: tags,
flowLimiter: cfdflow.NewLimiter(config.WarpRouting.MaxActiveFlows),
originDialerService: config.OriginDialerService,
log: log,
shutdownC: ctx.Done(),
}
if err := o.updateIngress(*config.Ingress, config.WarpRouting); err != nil {
return nil, err
@@ -175,7 +179,15 @@ func (o *Orchestrator) updateIngress(ingressRules ingress.Ingress, warpRouting i
// Update the flow limit since the configuration might have changed
o.flowLimiter.SetLimit(warpRouting.MaxActiveFlows)
proxy := proxy.NewOriginProxy(ingressRules, warpRouting, o.tags, o.flowLimiter, o.config.WriteTimeout, o.log)
// Update the origin dialer service with the new dialer settings
// We need to update the dialer here instead of creating a new instance of OriginDialerService because it has
// its own references and go routines. Specifically, the UDP dialer is a reference to this same service all the
// way into the datagram manager. Reconstructing the datagram manager is not something we currently provide during
// runtime in response to a configuration push except when starting a tunnel connection.
o.originDialerService.UpdateDefaultDialer(ingress.NewDialer(warpRouting))
// Create and replace the origin proxy with a new instance
proxy := proxy.NewOriginProxy(ingressRules, o.originDialerService, o.tags, o.flowLimiter, o.log)
o.proxy.Store(proxy)
o.config.Ingress = &ingressRules
o.config.WarpRouting = warpRouting
+96 -62
View File
@@ -41,6 +41,11 @@ var (
Value: "test",
},
}
testDefaultDialer = ingress.NewDialer(ingress.WarpRoutingConfig{
ConnectTimeout: config.CustomDuration{Duration: 1 * time.Second},
TCPKeepAlive: config.CustomDuration{Duration: 15 * time.Second},
MaxActiveFlows: 0,
})
)
// TestUpdateConfiguration tests that
@@ -50,10 +55,15 @@ var (
// - configurations can be deserialized
// - receiving an old version is noop
func TestUpdateConfiguration(t *testing.T) {
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &testLogger)
initConfig := &Config{
Ingress: &ingress.Ingress{},
Ingress: &ingress.Ingress{},
OriginDialerService: originDialer,
}
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, []ingress.Rule{ingress.NewManagementRule(management.New("management.argotunnel.com", false, "1.1.1.1:80", uuid.Nil, "", &testLogger, nil))}, &testLogger)
orchestrator, err := NewOrchestrator(t.Context(), initConfig, testTags, []ingress.Rule{ingress.NewManagementRule(management.New("management.argotunnel.com", false, "1.1.1.1:80", uuid.Nil, "", &testLogger, nil))}, &testLogger)
require.NoError(t, err)
initOriginProxy, err := orchestrator.GetOriginProxy()
require.NoError(t, err)
@@ -179,10 +189,15 @@ func TestUpdateConfiguration(t *testing.T) {
// Validates that a new version 0 will be applied if the configuration is loaded locally.
// This will happen when a locally managed tunnel is migrated to remote configuration and receives its first configuration.
func TestUpdateConfiguration_FromMigration(t *testing.T) {
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &testLogger)
initConfig := &Config{
Ingress: &ingress.Ingress{},
Ingress: &ingress.Ingress{},
OriginDialerService: originDialer,
}
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, []ingress.Rule{}, &testLogger)
orchestrator, err := NewOrchestrator(t.Context(), initConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
initOriginProxy, err := orchestrator.GetOriginProxy()
require.NoError(t, err)
@@ -205,10 +220,15 @@ func TestUpdateConfiguration_FromMigration(t *testing.T) {
// Validates that the default ingress rule will be set if there is no rule provided from the remote.
func TestUpdateConfiguration_WithoutIngressRule(t *testing.T) {
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &testLogger)
initConfig := &Config{
Ingress: &ingress.Ingress{},
Ingress: &ingress.Ingress{},
OriginDialerService: originDialer,
}
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, []ingress.Rule{}, &testLogger)
orchestrator, err := NewOrchestrator(t.Context(), initConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
initOriginProxy, err := orchestrator.GetOriginProxy()
require.NoError(t, err)
@@ -244,6 +264,11 @@ func TestConcurrentUpdateAndRead(t *testing.T) {
require.NoError(t, err)
defer tcpOrigin.Close()
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &testLogger)
var (
configJSONV1 = []byte(fmt.Sprintf(`
{
@@ -296,11 +321,12 @@ func TestConcurrentUpdateAndRead(t *testing.T) {
appliedV2 = make(chan struct{})
initConfig = &Config{
Ingress: &ingress.Ingress{},
Ingress: &ingress.Ingress{},
OriginDialerService: originDialer,
}
)
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
orchestrator, err := NewOrchestrator(ctx, initConfig, testTags, []ingress.Rule{}, &testLogger)
@@ -313,7 +339,7 @@ func TestConcurrentUpdateAndRead(t *testing.T) {
go func() {
serveTCPOrigin(t, tcpOrigin, &wg)
}()
for i := 0; i < concurrentRequests; i++ {
for i := range concurrentRequests {
originProxy, err := orchestrator.GetOriginProxy()
require.NoError(t, err)
wg.Add(1)
@@ -323,48 +349,37 @@ func TestConcurrentUpdateAndRead(t *testing.T) {
assert.NoError(t, err, "proxyHTTP %d failed %v", i, err)
defer resp.Body.Close()
var warpRoutingDisabled bool
// The response can be from initOrigin, http_status:204 or http_status:418
switch resp.StatusCode {
// v1 proxy, warp enabled
// v1 proxy
case 200:
body, err := io.ReadAll(resp.Body)
assert.NoError(t, err)
assert.Equal(t, t.Name(), string(body))
warpRoutingDisabled = false
// v2 proxy, warp disabled
// v2 proxy
case 204:
assert.Greater(t, i, concurrentRequests/4)
warpRoutingDisabled = true
// v3 proxy, warp enabled
// v3 proxy
case 418:
assert.Greater(t, i, concurrentRequests/2)
warpRoutingDisabled = false
}
// Once we have originProxy, it won't be changed by configuration updates.
// We can infer the version by the ProxyHTTP response code
pr, pw := io.Pipe()
w := newRespReadWriteFlusher()
// Write TCP message and make sure it's echo back. This has to be done in a go routune since ProxyTCP doesn't
// return until the stream is closed.
if !warpRoutingDisabled {
wg.Add(1)
go func() {
defer wg.Done()
defer pw.Close()
tcpEyeball(t, pw, tcpBody, w)
}()
}
wg.Add(1)
go func() {
defer wg.Done()
defer pw.Close()
tcpEyeball(t, pw, tcpBody, w)
}()
err = proxyTCP(ctx, originProxy, tcpOrigin.Addr().String(), w, pr)
if warpRoutingDisabled {
assert.Error(t, err, "expect proxyTCP %d to return error", i)
} else {
assert.NoError(t, err, "proxyTCP %d failed %v", i, err)
}
assert.NoError(t, err, "proxyTCP %d failed %v", i, err)
}(i, originProxy)
if i == concurrentRequests/4 {
@@ -393,7 +408,7 @@ func TestConcurrentUpdateAndRead(t *testing.T) {
// TestOverrideWarpRoutingConfigWithLocalValues tests that if a value is defined in the Config.ConfigurationFlags,
// it will override the value that comes from the remote result.
func TestOverrideWarpRoutingConfigWithLocalValues(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
assertMaxActiveFlows := func(orchestrator *Orchestrator, expectedValue uint64) {
@@ -406,39 +421,47 @@ func TestOverrideWarpRoutingConfigWithLocalValues(t *testing.T) {
require.EqualValues(t, expectedValue, warpRouting["maxActiveFlows"])
}
remoteValue := uint64(100)
remoteIngress := ingress.Ingress{}
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &testLogger)
// All the possible values set for MaxActiveFlows from the various points that can provide it:
// 1. Initialized value
// 2. Local CLI flag config
// 3. Remote configuration value
initValue := uint64(0)
localValue := uint64(100)
remoteValue := uint64(500)
initConfig := &Config{
Ingress: &ingress.Ingress{},
WarpRouting: ingress.WarpRoutingConfig{
MaxActiveFlows: initValue,
},
OriginDialerService: originDialer,
ConfigurationFlags: map[string]string{
flags.MaxActiveFlows: fmt.Sprintf("%d", localValue),
},
}
// We expect the local configuration flag to be the starting value
orchestrator, err := NewOrchestrator(ctx, initConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
assertMaxActiveFlows(orchestrator, localValue)
// Assigning the MaxActiveFlows in the remote config should be ignored over the local config
remoteWarpConfig := ingress.WarpRoutingConfig{
MaxActiveFlows: remoteValue,
}
remoteConfig := &Config{
Ingress: &remoteIngress,
WarpRouting: remoteWarpConfig,
ConfigurationFlags: map[string]string{},
}
orchestrator, err := NewOrchestrator(ctx, remoteConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
assertMaxActiveFlows(orchestrator, remoteValue)
// Add a local override for the maxActiveFlows
localValue := uint64(500)
remoteConfig.ConfigurationFlags[flags.MaxActiveFlows] = fmt.Sprintf("%d", localValue)
// Force a configuration refresh
err = orchestrator.updateIngress(remoteIngress, remoteWarpConfig)
err = orchestrator.updateIngress(ingress.Ingress{}, remoteWarpConfig)
require.NoError(t, err)
// Check the value being used is the local one
assertMaxActiveFlows(orchestrator, localValue)
// Remove local override for the maxActiveFlows
delete(remoteConfig.ConfigurationFlags, flags.MaxActiveFlows)
// Force a configuration refresh
err = orchestrator.updateIngress(remoteIngress, remoteWarpConfig)
require.NoError(t, err)
// Check the value being used is now the remote again
assertMaxActiveFlows(orchestrator, remoteValue)
}
func proxyHTTP(originProxy connection.OriginProxy, hostname string) (*http.Response, error) {
@@ -546,6 +569,10 @@ func updateWithValidation(t *testing.T, orchestrator *Orchestrator, version int3
// TestClosePreviousProxies makes sure proxies started in the previous configuration version are shutdown
func TestClosePreviousProxies(t *testing.T) {
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &testLogger)
var (
hostname = "hello.tunnel1.org"
configWithHelloWorld = []byte(fmt.Sprintf(`
@@ -576,11 +603,12 @@ func TestClosePreviousProxies(t *testing.T) {
}
`)
initConfig = &Config{
Ingress: &ingress.Ingress{},
Ingress: &ingress.Ingress{},
OriginDialerService: originDialer,
}
)
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
orchestrator, err := NewOrchestrator(ctx, initConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
@@ -638,10 +666,15 @@ func TestPersistentConnection(t *testing.T) {
hostname = "http://ws.tunnel.org"
)
msg := t.Name()
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &testLogger)
initConfig := &Config{
Ingress: &ingress.Ingress{},
Ingress: &ingress.Ingress{},
OriginDialerService: originDialer,
}
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, []ingress.Rule{}, &testLogger)
orchestrator, err := NewOrchestrator(t.Context(), initConfig, testTags, []ingress.Rule{}, &testLogger)
require.NoError(t, err)
wsOrigin := httptest.NewServer(http.HandlerFunc(wsEcho))
@@ -674,7 +707,7 @@ func TestPersistentConnection(t *testing.T) {
tcpReqReader, tcpReqWriter := io.Pipe()
tcpRespReadWriter := newRespReadWriteFlusher()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
var wg sync.WaitGroup
@@ -752,8 +785,9 @@ func TestSerializeLocalConfig(t *testing.T) {
ConfigurationFlags: map[string]string{"a": "b"},
}
result, _ := json.Marshal(c)
fmt.Println(string(result))
result, err := json.Marshal(c)
require.NoError(t, err)
require.JSONEq(t, `{"__configuration_flags":{"a":"b"},"ingress":[],"warp-routing":{"connectTimeout":0,"tcpKeepAlive":0}}`, string(result))
}
func wsEcho(w http.ResponseWriter, r *http.Request) {
+55 -17
View File
@@ -5,11 +5,11 @@ import (
"fmt"
"io"
"net/http"
"net/netip"
"strconv"
"time"
"github.com/pkg/errors"
pkgerrors "github.com/pkg/errors"
"github.com/rs/zerolog"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/trace"
@@ -35,7 +35,7 @@ const (
// Proxy represents a means to Proxy between cloudflared and the origin services.
type Proxy struct {
ingressRules ingress.Ingress
warpRouting *ingress.WarpRoutingService
originDialer ingress.OriginTCPDialer
tags []pogs.Tag
flowLimiter cfdflow.Limiter
log *zerolog.Logger
@@ -44,21 +44,19 @@ type Proxy struct {
// NewOriginProxy returns a new instance of the Proxy struct.
func NewOriginProxy(
ingressRules ingress.Ingress,
warpRouting ingress.WarpRoutingConfig,
originDialer ingress.OriginDialer,
tags []pogs.Tag,
flowLimiter cfdflow.Limiter,
writeTimeout time.Duration,
log *zerolog.Logger,
) *Proxy {
proxy := &Proxy{
ingressRules: ingressRules,
originDialer: originDialer,
tags: tags,
flowLimiter: flowLimiter,
log: log,
}
proxy.warpRouting = ingress.NewWarpRoutingService(warpRouting, writeTimeout)
return proxy
}
@@ -146,24 +144,18 @@ func (p *Proxy) ProxyHTTP(
// ProxyTCP proxies to a TCP connection between the origin service and cloudflared.
func (p *Proxy) ProxyTCP(
ctx context.Context,
rwa connection.ReadWriteAcker,
conn connection.ReadWriteAcker,
req *connection.TCPRequest,
) error {
incrementTCPRequests()
defer decrementTCPConcurrentRequests()
if p.warpRouting == nil {
err := errors.New(`cloudflared received a request from WARP client, but your configuration has disabled ingress from WARP clients. To enable this, set "warp-routing:\n\t enabled: true" in your config.yaml`)
p.log.Error().Msg(err.Error())
return err
}
logger := newTCPLogger(p.log, req)
// Try to start a new flow
if err := p.flowLimiter.Acquire(management.TCP.String()); err != nil {
logger.Warn().Msg("Too many concurrent flows being handled, rejecting tcp proxy")
return pkgerrors.Wrap(err, "failed to start tcp flow due to rate limiting")
return errors.Wrap(err, "failed to start tcp flow due to rate limiting")
}
defer p.flowLimiter.Release()
@@ -173,7 +165,14 @@ func (p *Proxy) ProxyTCP(
tracedCtx := tracing.NewTracedContext(serveCtx, req.CfTraceID, &logger)
logger.Debug().Msg("tcp proxy stream started")
if err := p.proxyStream(tracedCtx, rwa, req.Dest, p.warpRouting.Proxy, &logger); err != nil {
// Parse the destination into a netip.AddrPort
dest, err := netip.ParseAddrPort(req.Dest)
if err != nil {
logRequestError(&logger, err)
return err
}
if err := p.proxyTCPStream(tracedCtx, conn, dest, p.originDialer, &logger); err != nil {
logRequestError(&logger, err)
return err
}
@@ -279,14 +278,14 @@ func (p *Proxy) proxyStream(
tr *tracing.TracedContext,
rwa connection.ReadWriteAcker,
dest string,
connectionProxy ingress.StreamBasedOriginProxy,
originDialer ingress.StreamBasedOriginProxy,
logger *zerolog.Logger,
) error {
ctx := tr.Context
_, connectSpan := tr.Tracer().Start(ctx, "stream-connect")
start := time.Now()
originConn, err := connectionProxy.EstablishConnection(ctx, dest, logger)
originConn, err := originDialer.EstablishConnection(ctx, dest, logger)
if err != nil {
connectStreamErrors.Inc()
tracing.EndWithErrorStatus(connectSpan, err)
@@ -310,6 +309,45 @@ func (p *Proxy) proxyStream(
return nil
}
// proxyTCPStream proxies private network type TCP connections as a stream towards an available origin.
//
// This is different than proxyStream because it's not leveraged ingress rule services and uses the
// originDialer from OriginDialerService.
func (p *Proxy) proxyTCPStream(
tr *tracing.TracedContext,
tunnelConn connection.ReadWriteAcker,
dest netip.AddrPort,
originDialer ingress.OriginTCPDialer,
logger *zerolog.Logger,
) error {
ctx := tr.Context
_, connectSpan := tr.Tracer().Start(ctx, "stream-connect")
start := time.Now()
originConn, err := originDialer.DialTCP(ctx, dest)
if err != nil {
connectStreamErrors.Inc()
tracing.EndWithErrorStatus(connectSpan, err)
return err
}
connectSpan.End()
defer originConn.Close()
logger.Debug().Msg("origin connection established")
encodedSpans := tr.GetSpans()
if err := tunnelConn.AckConnection(encodedSpans); err != nil {
connectStreamErrors.Inc()
return err
}
connectLatency.Observe(float64(time.Since(start).Milliseconds()))
logger.Debug().Msg("proxy stream acknowledged")
stream.Pipe(tunnelConn, originConn, logger)
return nil
}
func (p *Proxy) proxyLocalRequest(proxy ingress.HTTPLocalProxy, w connection.ResponseWriter, req *http.Request, isWebsocket bool) {
if isWebsocket {
// These headers are added since they are stripped off during an eyeball request to origintunneld, but they
+43 -46
View File
@@ -33,17 +33,17 @@ import (
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/hello"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/tracing"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
var (
testTags = []pogs.Tag{{Name: "Name", Value: "value"}}
noWarpRouting = ingress.WarpRoutingConfig{}
testWarpRouting = ingress.WarpRoutingConfig{
ConnectTimeout: config.CustomDuration{Duration: time.Second},
}
testTags = []pogs.Tag{{Name: "Name", Value: "value"}}
testDefaultDialer = ingress.NewDialer(ingress.WarpRoutingConfig{
ConnectTimeout: config.CustomDuration{Duration: 1 * time.Second},
TCPKeepAlive: config.CustomDuration{Duration: 15 * time.Second},
MaxActiveFlows: 0,
})
)
type mockHTTPRespWriter struct {
@@ -148,7 +148,7 @@ func (w *mockSSERespWriter) ReadBytes() []byte {
func TestProxySingleOrigin(t *testing.T) {
log := zerolog.Nop()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
flagSet := flag.NewFlagSet(t.Name(), flag.PanicOnError)
flagSet.Bool("hello-world", true, "")
@@ -162,7 +162,12 @@ func TestProxySingleOrigin(t *testing.T) {
require.NoError(t, ingressRule.StartOrigins(&log, ctx.Done()))
proxy := NewOriginProxy(ingressRule, noWarpRouting, testTags, cfdflow.NewLimiter(0), time.Duration(0), &log)
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &log)
proxy := NewOriginProxy(ingressRule, originDialer, testTags, cfdflow.NewLimiter(0), &log)
t.Run("testProxyHTTP", testProxyHTTP(proxy))
t.Run("testProxyWebsocket", testProxyWebsocket(proxy))
t.Run("testProxySSE", testProxySSE(proxy))
@@ -190,7 +195,7 @@ func testProxyWebsocket(proxy connection.OriginProxy) func(t *testing.T) {
return func(t *testing.T) {
// WSRoute is a websocket echo handler
const testTimeout = 5 * time.Second * 1000
ctx, cancel := context.WithTimeout(context.Background(), testTimeout)
ctx, cancel := context.WithTimeout(t.Context(), testTimeout)
defer cancel()
readPipe, writePipe := io.Pipe()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("http://localhost:8080%s", hello.WSRoute), readPipe)
@@ -257,7 +262,7 @@ func testProxySSE(proxy connection.OriginProxy) func(t *testing.T) {
pushFreq = time.Millisecond * 10
)
responseWriter := newMockSSERespWriter()
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
req, err := http.NewRequestWithContext(ctx, http.MethodGet, fmt.Sprintf("http://localhost:8080%s?freq=%s", hello.SSERoute, pushFreq), nil)
require.NoError(t, err)
@@ -357,7 +362,7 @@ type MultipleIngressTest struct {
}
func runIngressTestScenarios(t *testing.T, unvalidatedIngress []config.UnvalidatedIngressRule, tests []MultipleIngressTest) {
ingress, err := ingress.ParseIngress(&config.Configuration{
ingressRule, err := ingress.ParseIngress(&config.Configuration{
TunnelID: t.Name(),
Ingress: unvalidatedIngress,
})
@@ -365,10 +370,15 @@ func runIngressTestScenarios(t *testing.T, unvalidatedIngress []config.Unvalidat
log := zerolog.Nop()
ctx, cancel := context.WithCancel(context.Background())
require.NoError(t, ingress.StartOrigins(&log, ctx.Done()))
ctx, cancel := context.WithCancel(t.Context())
require.NoError(t, ingressRule.StartOrigins(&log, ctx.Done()))
proxy := NewOriginProxy(ingress, noWarpRouting, testTags, cfdflow.NewLimiter(0), time.Duration(0), &log)
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &log)
proxy := NewOriginProxy(ingressRule, originDialer, testTags, cfdflow.NewLimiter(0), &log)
for _, test := range tests {
responseWriter := newMockHTTPRespWriter()
@@ -416,7 +426,12 @@ func TestProxyError(t *testing.T) {
log := zerolog.Nop()
proxy := NewOriginProxy(ing, noWarpRouting, testTags, cfdflow.NewLimiter(0), time.Duration(0), &log)
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 1 * time.Second,
}, &log)
proxy := NewOriginProxy(ing, originDialer, testTags, cfdflow.NewLimiter(0), &log)
responseWriter := newMockHTTPRespWriter()
req, err := http.NewRequest(http.MethodGet, "http://127.0.0.1", nil)
@@ -467,7 +482,7 @@ func (r *replayer) Bytes() []byte {
// WS - TCP: When a tcp based ingress is configured on the origin and the
// eyeball sends tcp packets wrapped in websockets. (E.g: cloudflared access).
func TestConnections(t *testing.T) {
logger := logger.Create(nil)
log := zerolog.Nop()
replayer := &replayer{rw: bytes.NewBuffer([]byte{})}
type args struct {
ingressServiceScheme string
@@ -475,9 +490,6 @@ func TestConnections(t *testing.T) {
eyeballResponseWriter connection.ResponseWriter
eyeballRequestBody io.ReadCloser
// Can be set to nil to show warp routing is not enabled.
warpRoutingService *ingress.WarpRoutingService
// eyeball connection type.
connectionType connection.Type
@@ -488,13 +500,18 @@ func TestConnections(t *testing.T) {
flowLimiterResponse error
}
originDialer := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
type want struct {
message []byte
headers http.Header
err bool
}
var tests = []struct {
tests := []struct {
name string
args args
want want
@@ -530,7 +547,6 @@ func TestConnections(t *testing.T) {
originService: runEchoTCPService,
eyeballResponseWriter: newTCPRespWriter(replayer),
eyeballRequestBody: newTCPRequestBody([]byte("test2")),
warpRoutingService: ingress.NewWarpRoutingService(testWarpRouting, time.Duration(0)),
connectionType: connection.TypeTCP,
requestHeaders: map[string][]string{
"Cf-Cloudflared-Proxy-Src": {"non-blank-value"},
@@ -548,7 +564,6 @@ func TestConnections(t *testing.T) {
originService: runEchoWSService,
// eyeballResponseWriter gets set after roundtrip dial.
eyeballRequestBody: newPipedWSRequestBody([]byte("test3")),
warpRoutingService: ingress.NewWarpRoutingService(testWarpRouting, time.Duration(0)),
requestHeaders: map[string][]string{
"Cf-Cloudflared-Proxy-Src": {"non-blank-value"},
},
@@ -601,23 +616,6 @@ func TestConnections(t *testing.T) {
headers: map[string][]string{},
},
},
{
name: "tcp-tcp proxy without warpRoutingService enabled",
args: args{
ingressServiceScheme: "tcp://",
originService: runEchoTCPService,
eyeballResponseWriter: newTCPRespWriter(replayer),
eyeballRequestBody: newTCPRequestBody([]byte("test2")),
connectionType: connection.TypeTCP,
requestHeaders: map[string][]string{
"Cf-Cloudflared-Proxy-Src": {"non-blank-value"},
},
},
want: want{
message: []byte{},
err: true,
},
},
{
name: "ws-ws proxy when origin is different",
args: args{
@@ -670,7 +668,6 @@ func TestConnections(t *testing.T) {
originService: runEchoTCPService,
eyeballResponseWriter: newTCPRespWriter(replayer),
eyeballRequestBody: newTCPRequestBody([]byte("rate-limited")),
warpRoutingService: ingress.NewWarpRoutingService(testWarpRouting, time.Duration(0)),
connectionType: connection.TypeTCP,
requestHeaders: map[string][]string{
"Cf-Cloudflared-Proxy-Src": {"non-blank-value"},
@@ -686,14 +683,14 @@ func TestConnections(t *testing.T) {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
ln, err := net.Listen("tcp", "127.0.0.1:0")
require.NoError(t, err)
// Starts origin service
test.args.originService(t, ln)
ingressRule := createSingleIngressConfig(t, test.args.ingressServiceScheme+ln.Addr().String())
_ = ingressRule.StartOrigins(logger, ctx.Done())
_ = ingressRule.StartOrigins(&log, ctx.Done())
// Mock flow limiter
ctrl := gomock.NewController(t)
@@ -702,8 +699,7 @@ func TestConnections(t *testing.T) {
flowLimiter.EXPECT().Acquire("tcp").AnyTimes().Return(test.args.flowLimiterResponse)
flowLimiter.EXPECT().Release().AnyTimes()
proxy := NewOriginProxy(ingressRule, testWarpRouting, testTags, flowLimiter, time.Duration(0), logger)
proxy.warpRouting = test.args.warpRoutingService
proxy := NewOriginProxy(ingressRule, originDialer, testTags, flowLimiter, &log)
dest := ln.Addr().String()
req, err := http.NewRequest(
@@ -755,6 +751,7 @@ func newWSRequestBody(data []byte) *requestBody {
pw: pw,
}
}
func newTCPRequestBody(data []byte) *requestBody {
pr, pw := io.Pipe()
go func() {
@@ -973,12 +970,12 @@ func runEchoTCPService(t *testing.T, l net.Listener) {
}
func runEchoWSService(t *testing.T, l net.Listener) {
var upgrader = gorillaWS.Upgrader{
upgrader := gorillaWS.Upgrader{
ReadBufferSize: 10,
WriteBufferSize: 10,
}
var ws = func(w http.ResponseWriter, r *http.Request) {
ws := func(w http.ResponseWriter, r *http.Request) {
header := make(http.Header)
for k, vs := range r.Header {
if k == "Test-Cloudflared-Echo" {
+30 -27
View File
@@ -11,12 +11,15 @@ import (
)
const (
namespace = "quic"
namespace = "quic"
ConnectionIndexMetricLabel = "conn_index"
frameTypeMetricLabel = "frame_type"
packetTypeMetricLabel = "packet_type"
reasonMetricLabel = "reason"
)
var (
clientConnLabels = []string{"conn_index"}
clientMetrics = struct {
clientMetrics = struct {
totalConnections prometheus.Counter
closedConnections prometheus.Counter
maxUDPPayloadSize *prometheus.GaugeVec
@@ -35,7 +38,7 @@ var (
congestionState *prometheus.GaugeVec
}{
totalConnections: prometheus.NewCounter(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "total_connections",
@@ -43,7 +46,7 @@ var (
},
),
closedConnections: prometheus.NewCounter(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "closed_connections",
@@ -57,70 +60,70 @@ var (
Name: "max_udp_payload",
Help: "Maximum UDP payload size in bytes for a QUIC packet",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
sentFrames: prometheus.NewCounterVec(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "sent_frames",
Help: "Number of frames that have been sent through a connection",
},
append(clientConnLabels, "frame_type"),
[]string{ConnectionIndexMetricLabel, frameTypeMetricLabel},
),
sentBytes: prometheus.NewCounterVec(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "sent_bytes",
Help: "Number of bytes that have been sent through a connection",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
receivedFrames: prometheus.NewCounterVec(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "received_frames",
Help: "Number of frames that have been received through a connection",
},
append(clientConnLabels, "frame_type"),
[]string{ConnectionIndexMetricLabel, frameTypeMetricLabel},
),
receivedBytes: prometheus.NewCounterVec(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "receive_bytes",
Help: "Number of bytes that have been received through a connection",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
bufferedPackets: prometheus.NewCounterVec(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "buffered_packets",
Help: "Number of bytes that have been buffered on a connection",
},
append(clientConnLabels, "packet_type"),
[]string{ConnectionIndexMetricLabel, packetTypeMetricLabel},
),
droppedPackets: prometheus.NewCounterVec(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "dropped_packets",
Help: "Number of bytes that have been dropped on a connection",
},
append(clientConnLabels, "packet_type", "reason"),
[]string{ConnectionIndexMetricLabel, packetTypeMetricLabel, reasonMetricLabel},
),
lostPackets: prometheus.NewCounterVec(
prometheus.CounterOpts{
prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "lost_packets",
Help: "Number of packets that have been lost from a connection",
},
append(clientConnLabels, "reason"),
[]string{ConnectionIndexMetricLabel, reasonMetricLabel},
),
minRTT: prometheus.NewGaugeVec(
prometheus.GaugeOpts{
@@ -129,7 +132,7 @@ var (
Name: "min_rtt",
Help: "Lowest RTT measured on a connection in millisec",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
latestRTT: prometheus.NewGaugeVec(
prometheus.GaugeOpts{
@@ -138,7 +141,7 @@ var (
Name: "latest_rtt",
Help: "Latest RTT measured on a connection",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
smoothedRTT: prometheus.NewGaugeVec(
prometheus.GaugeOpts{
@@ -147,7 +150,7 @@ var (
Name: "smoothed_rtt",
Help: "Calculated smoothed RTT measured on a connection in millisec",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
mtu: prometheus.NewGaugeVec(
prometheus.GaugeOpts{
@@ -156,7 +159,7 @@ var (
Name: "mtu",
Help: "Current maximum transmission unit (MTU) of a connection",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
congestionWindow: prometheus.NewGaugeVec(
prometheus.GaugeOpts{
@@ -165,7 +168,7 @@ var (
Name: "congestion_window",
Help: "Current congestion window size",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
congestionState: prometheus.NewGaugeVec(
prometheus.GaugeOpts{
@@ -174,13 +177,13 @@ var (
Name: "congestion_state",
Help: "Current congestion control state. See https://pkg.go.dev/github.com/quic-go/quic-go@v0.45.0/logging#CongestionState for what each value maps to",
},
clientConnLabels,
[]string{ConnectionIndexMetricLabel},
),
}
registerClient = sync.Once{}
packetTooBigDropped = prometheus.NewCounter(prometheus.CounterOpts{
packetTooBigDropped = prometheus.NewCounter(prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: "client",
Name: "packet_too_big_dropped",
+4 -7
View File
@@ -2,12 +2,11 @@ package v3
import (
"errors"
"net"
"net/netip"
"sync"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/management"
cfdflow "github.com/cloudflare/cloudflared/flow"
@@ -38,18 +37,16 @@ type SessionManager interface {
UnregisterSession(requestID RequestID)
}
type DialUDP func(dest netip.AddrPort) (*net.UDPConn, error)
type sessionManager struct {
sessions map[RequestID]Session
mutex sync.RWMutex
originDialer DialUDP
originDialer ingress.OriginUDPDialer
limiter cfdflow.Limiter
metrics Metrics
log *zerolog.Logger
}
func NewSessionManager(metrics Metrics, log *zerolog.Logger, originDialer DialUDP, limiter cfdflow.Limiter) SessionManager {
func NewSessionManager(metrics Metrics, log *zerolog.Logger, originDialer ingress.OriginUDPDialer, limiter cfdflow.Limiter) SessionManager {
return &sessionManager{
sessions: make(map[RequestID]Session),
originDialer: originDialer,
@@ -76,7 +73,7 @@ func (s *sessionManager) RegisterSession(request *UDPSessionRegistrationDatagram
}
// Attempt to bind the UDP socket for the new session
origin, err := s.originDialer(request.Dest)
origin, err := s.originDialer.DialUDP(request.Dest)
if err != nil {
return nil, err
}
+24 -3
View File
@@ -11,6 +11,7 @@ import (
"github.com/stretchr/testify/require"
"go.uber.org/mock/gomock"
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/mocks"
cfdflow "github.com/cloudflare/cloudflared/flow"
@@ -18,9 +19,21 @@ import (
v3 "github.com/cloudflare/cloudflared/quic/v3"
)
var (
testDefaultDialer = ingress.NewDialer(ingress.WarpRoutingConfig{
ConnectTimeout: config.CustomDuration{Duration: 1 * time.Second},
TCPKeepAlive: config.CustomDuration{Duration: 15 * time.Second},
MaxActiveFlows: 0,
})
)
func TestRegisterSession(t *testing.T) {
log := zerolog.Nop()
manager := v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, cfdflow.NewLimiter(0))
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
manager := v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, cfdflow.NewLimiter(0))
request := v3.UDPSessionRegistrationDatagram{
RequestID: testRequestID,
@@ -76,7 +89,11 @@ func TestRegisterSession(t *testing.T) {
func TestGetSession_Empty(t *testing.T) {
log := zerolog.Nop()
manager := v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, cfdflow.NewLimiter(0))
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
manager := v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, cfdflow.NewLimiter(0))
_, err := manager.GetSession(testRequestID)
if !errors.Is(err, v3.ErrSessionNotFound) {
@@ -86,6 +103,10 @@ func TestGetSession_Empty(t *testing.T) {
func TestRegisterSessionRateLimit(t *testing.T) {
log := zerolog.Nop()
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
ctrl := gomock.NewController(t)
flowLimiterMock := mocks.NewMockLimiter(ctrl)
@@ -93,7 +114,7 @@ func TestRegisterSessionRateLimit(t *testing.T) {
flowLimiterMock.EXPECT().Acquire("udp").Return(cfdflow.ErrTooManyActiveFlows)
flowLimiterMock.EXPECT().Release().Times(0)
manager := v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, flowLimiterMock)
manager := v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, flowLimiterMock)
request := v3.UDPSessionRegistrationDatagram{
RequestID: testRequestID,
+50 -31
View File
@@ -1,83 +1,101 @@
package v3
import (
"fmt"
"github.com/prometheus/client_golang/prometheus"
"github.com/cloudflare/cloudflared/quic"
)
const (
namespace = "cloudflared"
subsystem = "udp"
commandMetricLabel = "command"
)
type Metrics interface {
IncrementFlows()
DecrementFlows()
PayloadTooLarge()
RetryFlowResponse()
MigrateFlow()
IncrementFlows(connIndex uint8)
DecrementFlows(connIndex uint8)
PayloadTooLarge(connIndex uint8)
RetryFlowResponse(connIndex uint8)
MigrateFlow(connIndex uint8)
UnsupportedRemoteCommand(connIndex uint8, command string)
}
type metrics struct {
activeUDPFlows prometheus.Gauge
totalUDPFlows prometheus.Counter
payloadTooLarge prometheus.Counter
retryFlowResponses prometheus.Counter
migratedFlows prometheus.Counter
activeUDPFlows *prometheus.GaugeVec
totalUDPFlows *prometheus.CounterVec
payloadTooLarge *prometheus.CounterVec
retryFlowResponses *prometheus.CounterVec
migratedFlows *prometheus.CounterVec
unsupportedRemoteCommands *prometheus.CounterVec
}
func (m *metrics) IncrementFlows() {
m.totalUDPFlows.Inc()
m.activeUDPFlows.Inc()
func (m *metrics) IncrementFlows(connIndex uint8) {
m.totalUDPFlows.WithLabelValues(fmt.Sprintf("%d", connIndex)).Inc()
m.activeUDPFlows.WithLabelValues(fmt.Sprintf("%d", connIndex)).Inc()
}
func (m *metrics) DecrementFlows() {
m.activeUDPFlows.Dec()
func (m *metrics) DecrementFlows(connIndex uint8) {
m.activeUDPFlows.WithLabelValues(fmt.Sprintf("%d", connIndex)).Dec()
}
func (m *metrics) PayloadTooLarge() {
m.payloadTooLarge.Inc()
func (m *metrics) PayloadTooLarge(connIndex uint8) {
m.payloadTooLarge.WithLabelValues(fmt.Sprintf("%d", connIndex)).Inc()
}
func (m *metrics) RetryFlowResponse() {
m.retryFlowResponses.Inc()
func (m *metrics) RetryFlowResponse(connIndex uint8) {
m.retryFlowResponses.WithLabelValues(fmt.Sprintf("%d", connIndex)).Inc()
}
func (m *metrics) MigrateFlow() {
m.migratedFlows.Inc()
func (m *metrics) MigrateFlow(connIndex uint8) {
m.migratedFlows.WithLabelValues(fmt.Sprintf("%d", connIndex)).Inc()
}
func (m *metrics) UnsupportedRemoteCommand(connIndex uint8, command string) {
m.unsupportedRemoteCommands.WithLabelValues(fmt.Sprintf("%d", connIndex), command).Inc()
}
func NewMetrics(registerer prometheus.Registerer) Metrics {
m := &metrics{
activeUDPFlows: prometheus.NewGauge(prometheus.GaugeOpts{
activeUDPFlows: prometheus.NewGaugeVec(prometheus.GaugeOpts{
Namespace: namespace,
Subsystem: subsystem,
Name: "active_flows",
Help: "Concurrent count of UDP flows that are being proxied to any origin",
}),
totalUDPFlows: prometheus.NewCounter(prometheus.CounterOpts{
}, []string{quic.ConnectionIndexMetricLabel}),
totalUDPFlows: prometheus.NewCounterVec(prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: subsystem,
Name: "total_flows",
Help: "Total count of UDP flows that have been proxied to any origin",
}),
payloadTooLarge: prometheus.NewCounter(prometheus.CounterOpts{
}, []string{quic.ConnectionIndexMetricLabel}),
payloadTooLarge: prometheus.NewCounterVec(prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: subsystem,
Name: "payload_too_large",
Help: "Total count of UDP flows that have had origin payloads that are too large to proxy",
}),
retryFlowResponses: prometheus.NewCounter(prometheus.CounterOpts{
}, []string{quic.ConnectionIndexMetricLabel}),
retryFlowResponses: prometheus.NewCounterVec(prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: subsystem,
Name: "retry_flow_responses",
Help: "Total count of UDP flows that have had to send their registration response more than once",
}),
migratedFlows: prometheus.NewCounter(prometheus.CounterOpts{
}, []string{quic.ConnectionIndexMetricLabel}),
migratedFlows: prometheus.NewCounterVec(prometheus.CounterOpts{ //nolint:promlinter
Namespace: namespace,
Subsystem: subsystem,
Name: "migrated_flows",
Help: "Total count of UDP flows have been migrated across local connections",
}),
}, []string{quic.ConnectionIndexMetricLabel}),
unsupportedRemoteCommands: prometheus.NewCounterVec(prometheus.CounterOpts{
Namespace: namespace,
Subsystem: subsystem,
Name: "unsupported_remote_command_total",
Help: "Total count of unsupported remote RPC commands for the ",
}, []string{quic.ConnectionIndexMetricLabel, commandMetricLabel}),
}
registerer.MustRegister(
m.activeUDPFlows,
@@ -85,6 +103,7 @@ func NewMetrics(registerer prometheus.Registerer) Metrics {
m.payloadTooLarge,
m.retryFlowResponses,
m.migratedFlows,
m.unsupportedRemoteCommands,
)
return m
}
+6 -5
View File
@@ -2,8 +2,9 @@ package v3_test
type noopMetrics struct{}
func (noopMetrics) IncrementFlows() {}
func (noopMetrics) DecrementFlows() {}
func (noopMetrics) PayloadTooLarge() {}
func (noopMetrics) RetryFlowResponse() {}
func (noopMetrics) MigrateFlow() {}
func (noopMetrics) IncrementFlows(connIndex uint8) {}
func (noopMetrics) DecrementFlows(connIndex uint8) {}
func (noopMetrics) PayloadTooLarge(connIndex uint8) {}
func (noopMetrics) RetryFlowResponse(connIndex uint8) {}
func (noopMetrics) MigrateFlow(connIndex uint8) {}
func (noopMetrics) UnsupportedRemoteCommand(connIndex uint8, command string) {}
+3 -3
View File
@@ -264,10 +264,10 @@ func (c *datagramConn) handleSessionRegistrationDatagram(ctx context.Context, da
return
}
log = log.With().Str(logSrcKey, session.LocalAddr().String()).Logger()
c.metrics.IncrementFlows()
c.metrics.IncrementFlows(c.index)
// Make sure to eventually remove the session from the session manager when the session is closed
defer c.sessionManager.UnregisterSession(session.ID())
defer c.metrics.DecrementFlows()
defer c.metrics.DecrementFlows(c.index)
// Respond that we are able to process the new session
err = c.SendUDPSessionResponse(datagram.RequestID, ResponseOk)
@@ -315,7 +315,7 @@ func (c *datagramConn) handleSessionAlreadyRegistered(requestID RequestID, logge
// The session is already running in another routine so we want to restart the idle timeout since no proxied
// packets have come down yet.
session.ResetIdleTimer()
c.metrics.RetryFlowResponse()
c.metrics.RetryFlowResponse(c.index)
logger.Debug().Msgf("flow registration response retry")
}
+48 -24
View File
@@ -88,7 +88,11 @@ func (m *mockEyeball) SendICMPTTLExceed(icmp *packet.ICMP, rawPacket packet.RawP
func TestDatagramConn_New(t *testing.T) {
log := zerolog.Nop()
conn := v3.NewDatagramConn(newMockQuicConn(), v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
conn := v3.NewDatagramConn(newMockQuicConn(), v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
if conn == nil {
t.Fatal("expected valid connection")
}
@@ -96,8 +100,12 @@ func TestDatagramConn_New(t *testing.T) {
func TestDatagramConn_SendUDPSessionDatagram(t *testing.T) {
log := zerolog.Nop()
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
quic := newMockQuicConn()
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
payload := []byte{0xef, 0xef}
err := conn.SendUDPSessionDatagram(payload)
@@ -111,8 +119,12 @@ func TestDatagramConn_SendUDPSessionDatagram(t *testing.T) {
func TestDatagramConn_SendUDPSessionResponse(t *testing.T) {
log := zerolog.Nop()
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
quic := newMockQuicConn()
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
err := conn.SendUDPSessionResponse(testRequestID, v3.ResponseDestinationUnreachable)
require.NoError(t, err)
@@ -133,10 +145,14 @@ func TestDatagramConn_SendUDPSessionResponse(t *testing.T) {
func TestDatagramConnServe_ApplicationClosed(t *testing.T) {
log := zerolog.Nop()
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
quic := newMockQuicConn()
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
ctx, cancel := context.WithTimeout(t.Context(), 1*time.Second)
defer cancel()
err := conn.Serve(ctx)
if !errors.Is(err, context.DeadlineExceeded) {
@@ -146,13 +162,17 @@ func TestDatagramConnServe_ApplicationClosed(t *testing.T) {
func TestDatagramConnServe_ConnectionClosed(t *testing.T) {
log := zerolog.Nop()
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
quic := newMockQuicConn()
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
ctx, cancel := context.WithTimeout(t.Context(), 1*time.Second)
defer cancel()
quic.ctx = ctx
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
err := conn.Serve(context.Background())
err := conn.Serve(t.Context())
if !errors.Is(err, context.DeadlineExceeded) {
t.Fatal(err)
}
@@ -160,10 +180,14 @@ func TestDatagramConnServe_ConnectionClosed(t *testing.T) {
func TestDatagramConnServe_ReceiveDatagramError(t *testing.T) {
log := zerolog.Nop()
originDialerService := ingress.NewOriginDialer(ingress.OriginConfig{
DefaultDialer: testDefaultDialer,
TCPWriteTimeout: 0,
}, &log)
quic := &mockQuicConnReadError{err: net.ErrClosed}
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, ingress.DialUDPAddrPort, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
conn := v3.NewDatagramConn(quic, v3.NewSessionManager(&noopMetrics{}, &log, originDialerService, cfdflow.NewLimiter(0)), &noopICMPRouter{}, 0, &noopMetrics{}, &log)
err := conn.Serve(context.Background())
err := conn.Serve(t.Context())
if !errors.Is(err, net.ErrClosed) {
t.Fatal(err)
}
@@ -178,7 +202,7 @@ func TestDatagramConnServe_SessionRegistrationRateLimit(t *testing.T) {
conn := v3.NewDatagramConn(quic, sessionManager, &noopICMPRouter{}, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithCancel(t.Context())
defer cancel()
done := make(chan error, 1)
go func() {
@@ -230,7 +254,7 @@ func TestDatagramConnServe_ErrorDatagramTypes(t *testing.T) {
quic.send <- test.input
conn := v3.NewDatagramConn(quic, &mockSessionManager{}, &noopICMPRouter{}, 0, &noopMetrics{}, &log)
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
ctx, cancel := context.WithTimeout(t.Context(), 1*time.Second)
defer cancel()
err := conn.Serve(ctx)
// we cancel the Serve method to check to see if the log output was written since the unsupported datagram
@@ -272,7 +296,7 @@ func TestDatagramConnServe_RegisterSession_SessionManagerError(t *testing.T) {
conn := v3.NewDatagramConn(quic, &sessionManager, &noopICMPRouter{}, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
@@ -307,7 +331,7 @@ func TestDatagramConnServe(t *testing.T) {
conn := v3.NewDatagramConn(quic, &sessionManager, &noopICMPRouter{}, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
@@ -356,7 +380,7 @@ func TestDatagramConnServeDecodeMultipleICMPInParallel(t *testing.T) {
conn := v3.NewDatagramConn(quic, &sessionManager, router, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
@@ -435,7 +459,7 @@ func TestDatagramConnServe_RegisterTwice(t *testing.T) {
conn := v3.NewDatagramConn(quic, &sessionManager, &noopICMPRouter{}, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
@@ -499,14 +523,14 @@ func TestDatagramConnServe_MigrateConnection(t *testing.T) {
conn2 := v3.NewDatagramConn(quic2, &sessionManager, &noopICMPRouter{}, 1, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
done <- conn.Serve(ctx)
}()
ctx2, cancel2 := context.WithCancelCause(context.Background())
ctx2, cancel2 := context.WithCancelCause(t.Context())
defer cancel2(errors.New("other error"))
done2 := make(chan error, 1)
go func() {
@@ -580,7 +604,7 @@ func TestDatagramConnServe_Payload_GetSessionError(t *testing.T) {
conn := v3.NewDatagramConn(quic, &sessionManager, &noopICMPRouter{}, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
@@ -608,7 +632,7 @@ func TestDatagramConnServe_Payload(t *testing.T) {
conn := v3.NewDatagramConn(quic, &sessionManager, &noopICMPRouter{}, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
@@ -637,7 +661,7 @@ func TestDatagramConnServe_ICMPDatagram_TTLDecremented(t *testing.T) {
conn := v3.NewDatagramConn(quic, &mockSessionManager{}, router, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
@@ -683,7 +707,7 @@ func TestDatagramConnServe_ICMPDatagram_TTLExceeded(t *testing.T) {
conn := v3.NewDatagramConn(quic, &mockSessionManager{}, router, 0, &noopMetrics{}, &log)
// Setup the muxer
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(errors.New("other error"))
done := make(chan error, 1)
go func() {
@@ -781,12 +805,12 @@ func newICMPDatagram(pk *packet.ICMP) []byte {
// Cancel the provided context and make sure it closes with the expected cancellation error
func assertContextClosed(t *testing.T, ctx context.Context, done <-chan error, cancel context.CancelCauseFunc) {
cancel(expectedContextCanceled)
cancel(errExpectedContextCanceled)
err := <-done
if !errors.Is(err, context.Canceled) {
t.Fatal(err)
}
if !errors.Is(context.Cause(ctx), expectedContextCanceled) {
if !errors.Is(context.Cause(ctx), errExpectedContextCanceled) {
t.Fatal(err)
}
}
+8 -6
View File
@@ -27,11 +27,11 @@ const (
)
// SessionCloseErr indicates that the session's Close method was called.
var SessionCloseErr error = errors.New("flow was closed directly")
var SessionCloseErr error = errors.New("flow was closed directly") //nolint:errname
// SessionIdleErr is returned when the session was closed because there was no communication
// in either direction over the session for the timeout period.
type SessionIdleErr struct {
type SessionIdleErr struct { //nolint:errname
timeout time.Duration
}
@@ -149,7 +149,8 @@ func (s *session) Migrate(eyeball DatagramConn, ctx context.Context, logger *zer
}
// The session is already running so we want to restart the idle timeout since no proxied packets have come down yet.
s.markActive()
s.metrics.MigrateFlow()
connectionIndex := eyeball.ID()
s.metrics.MigrateFlow(connectionIndex)
}
func (s *session) Serve(ctx context.Context) error {
@@ -160,7 +161,7 @@ func (s *session) Serve(ctx context.Context) error {
// To perform a zero copy write when passing the datagram to the connection, we prepare the buffer with
// the required datagram header information. We can reuse this buffer for this session since the header is the
// same for the each read.
MarshalPayloadHeaderTo(s.id, readBuffer[:DatagramPayloadHeaderLen])
_ = MarshalPayloadHeaderTo(s.id, readBuffer[:DatagramPayloadHeaderLen])
for {
// Read from the origin UDP socket
n, err := s.origin.Read(readBuffer[DatagramPayloadHeaderLen:])
@@ -177,7 +178,8 @@ func (s *session) Serve(ctx context.Context) error {
continue
}
if n > maxDatagramPayloadLen {
s.metrics.PayloadTooLarge()
connectionIndex := s.ConnectionID()
s.metrics.PayloadTooLarge(connectionIndex)
s.log.Error().Int(logPacketSizeKey, n).Msg("flow (origin) packet read was too large and was dropped")
continue
}
@@ -241,7 +243,7 @@ func (s *session) waitForCloseCondition(ctx context.Context, closeAfterIdle time
// Closing the session at the end cancels read so Serve() can return
defer s.Close()
if closeAfterIdle == 0 {
// provide deafult is caller doesn't specify one
// Provided that the default caller doesn't specify one
closeAfterIdle = defaultCloseIdleAfter
}
+17 -17
View File
@@ -17,7 +17,7 @@ import (
)
var (
expectedContextCanceled = errors.New("expected context canceled")
errExpectedContextCanceled = errors.New("expected context canceled")
testOriginAddr = net.UDPAddrFromAddrPort(netip.MustParseAddrPort("127.0.0.1:0"))
testLocalAddr = net.UDPAddrFromAddrPort(netip.MustParseAddrPort("127.0.0.1:0"))
@@ -40,7 +40,7 @@ func testSessionWrite(t *testing.T, payload []byte) {
serverRead := make(chan []byte, 1)
go func() {
read := make([]byte, 1500)
server.Read(read[:])
_, _ = server.Read(read[:])
serverRead <- read
}()
// Create session and write to origin
@@ -93,7 +93,7 @@ func testSessionServe_Origin(t *testing.T, payload []byte) {
session := v3.NewSession(testRequestID, 3*time.Second, origin, testOriginAddr, testLocalAddr, &eyeball, &noopMetrics{}, &log)
defer session.Close()
ctx, cancel := context.WithCancelCause(context.Background())
ctx, cancel := context.WithCancelCause(t.Context())
defer cancel(context.Canceled)
done := make(chan error)
go func() {
@@ -110,12 +110,12 @@ func testSessionServe_Origin(t *testing.T, payload []byte) {
case data := <-eyeball.recvData:
// check received data matches provided from origin
expectedData := makePayload(1500)
v3.MarshalPayloadHeaderTo(testRequestID, expectedData[:])
_ = v3.MarshalPayloadHeaderTo(testRequestID, expectedData[:])
copy(expectedData[17:], payload)
if !slices.Equal(expectedData[:v3.DatagramPayloadHeaderLen+len(payload)], data) {
t.Fatal("expected datagram did not equal expected")
}
cancel(expectedContextCanceled)
cancel(errExpectedContextCanceled)
case err := <-ctx.Done():
// we expect the payload to return before the context to cancel on the session
t.Fatal(err)
@@ -125,7 +125,7 @@ func testSessionServe_Origin(t *testing.T, payload []byte) {
if !errors.Is(err, context.Canceled) {
t.Fatal(err)
}
if !errors.Is(context.Cause(ctx), expectedContextCanceled) {
if !errors.Is(context.Cause(ctx), errExpectedContextCanceled) {
t.Fatal(err)
}
}
@@ -143,7 +143,7 @@ func TestSessionServe_OriginTooLarge(t *testing.T) {
done := make(chan error)
go func() {
done <- session.Serve(context.Background())
done <- session.Serve(t.Context())
}()
// Attempt to write a payload too large from the origin
@@ -173,7 +173,7 @@ func TestSessionServe_Migrate(t *testing.T) {
defer session.Close()
done := make(chan error)
eyeball1Ctx, cancel := context.WithCancelCause(context.Background())
eyeball1Ctx, cancel := context.WithCancelCause(t.Context())
go func() {
done <- session.Serve(eyeball1Ctx)
}()
@@ -181,7 +181,7 @@ func TestSessionServe_Migrate(t *testing.T) {
// Migrate the session to a new connection before origin sends data
eyeball2 := newMockEyeball()
eyeball2.connID = 1
eyeball2Ctx := context.Background()
eyeball2Ctx := t.Context()
session.Migrate(&eyeball2, eyeball2Ctx, &log)
// Cancel the origin eyeball context; this should not cancel the session
@@ -198,7 +198,7 @@ func TestSessionServe_Migrate(t *testing.T) {
// Origin sends data
payload2 := []byte{0xde}
pipe1.Write(payload2)
_, _ = pipe1.Write(payload2)
// Expect write to eyeball2
data := <-eyeball2.recvData
@@ -230,7 +230,7 @@ func TestSessionServe_Migrate_CloseContext2(t *testing.T) {
defer session.Close()
done := make(chan error)
eyeball1Ctx, cancel := context.WithCancelCause(context.Background())
eyeball1Ctx, cancel := context.WithCancelCause(t.Context())
go func() {
done <- session.Serve(eyeball1Ctx)
}()
@@ -238,7 +238,7 @@ func TestSessionServe_Migrate_CloseContext2(t *testing.T) {
// Migrate the session to a new connection before origin sends data
eyeball2 := newMockEyeball()
eyeball2.connID = 1
eyeball2Ctx, cancel2 := context.WithCancelCause(context.Background())
eyeball2Ctx, cancel2 := context.WithCancelCause(t.Context())
session.Migrate(&eyeball2, eyeball2Ctx, &log)
// Cancel the origin eyeball context; this should not cancel the session
@@ -249,13 +249,13 @@ func TestSessionServe_Migrate_CloseContext2(t *testing.T) {
t.Fatalf("expected session to still be running")
default:
}
if context.Cause(eyeball1Ctx) != contextCancelErr {
if !errors.Is(context.Cause(eyeball1Ctx), contextCancelErr) {
t.Fatalf("first eyeball context should be cancelled manually: %+v", context.Cause(eyeball1Ctx))
}
// Origin sends data
payload2 := []byte{0xde}
pipe1.Write(payload2)
_, _ = pipe1.Write(payload2)
// Expect write to eyeball2
data := <-eyeball2.recvData
@@ -316,7 +316,7 @@ func TestSessionServe_IdleTimeout(t *testing.T) {
defer server.Close()
closeAfterIdle := 2 * time.Second
session := v3.NewSession(testRequestID, closeAfterIdle, origin, testOriginAddr, testLocalAddr, &noopEyeball{}, &noopMetrics{}, &log)
err := session.Serve(context.Background())
err := session.Serve(t.Context())
if !errors.Is(err, v3.SessionIdleErr{}) {
t.Fatal(err)
}
@@ -342,7 +342,7 @@ func TestSessionServe_ParentContextCanceled(t *testing.T) {
closeAfterIdle := 10 * time.Second
session := v3.NewSession(testRequestID, closeAfterIdle, origin, testOriginAddr, testLocalAddr, &noopEyeball{}, &noopMetrics{}, &log)
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
ctx, cancel := context.WithTimeout(t.Context(), 2*time.Second)
defer cancel()
err := session.Serve(ctx)
if !errors.Is(err, context.DeadlineExceeded) {
@@ -366,7 +366,7 @@ func TestSessionServe_ReadErrors(t *testing.T) {
log := zerolog.Nop()
origin := newTestErrOrigin(net.ErrClosed, nil)
session := v3.NewSession(testRequestID, 30*time.Second, &origin, testOriginAddr, testLocalAddr, &noopEyeball{}, &noopMetrics{}, &log)
err := session.Serve(context.Background())
err := session.Serve(t.Context())
if !errors.Is(err, net.ErrClosed) {
t.Fatal(err)
}
+2 -2
View File
@@ -17,8 +17,8 @@ const (
)
var (
nonFipsPostQuantumStrictPKex []tls.CurveID = []tls.CurveID{X25519MLKEM768PQKex, X25519Kyber768Draft00PQKex}
nonFipsPostQuantumPreferPKex []tls.CurveID = []tls.CurveID{X25519MLKEM768PQKex, X25519Kyber768Draft00PQKex}
nonFipsPostQuantumStrictPKex []tls.CurveID = []tls.CurveID{X25519MLKEM768PQKex}
nonFipsPostQuantumPreferPKex []tls.CurveID = []tls.CurveID{X25519MLKEM768PQKex}
fipsPostQuantumStrictPKex []tls.CurveID = []tls.CurveID{P256Kyber768Draft00PQKex}
fipsPostQuantumPreferPKex []tls.CurveID = []tls.CurveID{P256Kyber768Draft00PQKex, tls.CurveP256}
)
+38 -3
View File
@@ -2,12 +2,16 @@ package supervisor
import (
"crypto/tls"
"net/http"
"net/http/httptest"
"slices"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/fips"
)
func TestCurvePreferences(t *testing.T) {
@@ -48,7 +52,7 @@ func TestCurvePreferences(t *testing.T) {
pqMode: features.PostQuantumPrefer,
fipsEnabled: false,
currentCurves: []tls.CurveID{tls.CurveP256},
expectedCurves: []tls.CurveID{X25519MLKEM768PQKex, X25519Kyber768Draft00PQKex, tls.CurveP256},
expectedCurves: []tls.CurveID{X25519MLKEM768PQKex, tls.CurveP256},
},
{
name: "Non FIPS with Prefer PQ - no duplicates",
@@ -62,14 +66,14 @@ func TestCurvePreferences(t *testing.T) {
pqMode: features.PostQuantumPrefer,
fipsEnabled: false,
currentCurves: []tls.CurveID{tls.CurveP256, X25519Kyber768Draft00PQKex},
expectedCurves: []tls.CurveID{X25519MLKEM768PQKex, X25519Kyber768Draft00PQKex, tls.CurveP256},
expectedCurves: []tls.CurveID{X25519MLKEM768PQKex, tls.CurveP256, X25519Kyber768Draft00PQKex},
},
{
name: "Non FIPS with Strict PQ",
pqMode: features.PostQuantumStrict,
fipsEnabled: false,
currentCurves: []tls.CurveID{tls.CurveP256},
expectedCurves: []tls.CurveID{X25519MLKEM768PQKex, X25519Kyber768Draft00PQKex},
expectedCurves: []tls.CurveID{X25519MLKEM768PQKex},
},
}
@@ -82,3 +86,34 @@ func TestCurvePreferences(t *testing.T) {
})
}
}
func runClientServerHandshake(t *testing.T, curves []tls.CurveID) []tls.CurveID {
var advertisedCurves []tls.CurveID
ts := httptest.NewUnstartedServer(nil)
ts.TLS = &tls.Config{ // nolint: gosec
GetConfigForClient: func(chi *tls.ClientHelloInfo) (*tls.Config, error) {
advertisedCurves = slices.Clone(chi.SupportedCurves)
return nil, nil
},
}
ts.StartTLS()
defer ts.Close()
clientTlsConfig := ts.Client().Transport.(*http.Transport).TLSClientConfig
clientTlsConfig.CurvePreferences = curves
resp, err := ts.Client().Head(ts.URL)
if err != nil {
t.Error(err)
return nil
}
defer resp.Body.Close()
return advertisedCurves
}
func TestSupportedCurvesNegotiation(t *testing.T) {
for _, tcase := range []features.PostQuantumMode{features.PostQuantumPrefer} {
curves, err := curvePreference(tcase, fips.IsFipsEnabled(), make([]tls.CurveID, 0))
require.NoError(t, err)
advertisedCurves := runClientServerHandshake(t, curves)
assert.Equal(t, curves, advertisedCurves)
}
}
+5 -2
View File
@@ -13,7 +13,6 @@ import (
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/edgediscovery"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/orchestration"
v3 "github.com/cloudflare/cloudflared/quic/v3"
"github.com/cloudflare/cloudflared/retry"
@@ -78,7 +77,8 @@ func NewSupervisor(config *TunnelConfig, orchestrator *orchestration.Orchestrato
edgeBindAddr := config.EdgeBindAddr
datagramMetrics := v3.NewMetrics(prometheus.DefaultRegisterer)
sessionManager := v3.NewSessionManager(datagramMetrics, config.Log, ingress.DialUDPAddrPort, orchestrator.GetFlowLimiter())
sessionManager := v3.NewSessionManager(datagramMetrics, config.Log, config.OriginDialerService, orchestrator.GetFlowLimiter())
edgeTunnelServer := EdgeTunnelServer{
config: config,
@@ -125,6 +125,9 @@ func (s *Supervisor) Run(
}()
}
// Setup DNS Resolver refresh
go s.config.OriginDNSService.StartRefreshLoop(ctx)
if err := s.initialize(ctx, connectedSignal); err != nil {
if err == errEarlyShutdown {
return nil
+25 -27
View File
@@ -17,12 +17,14 @@ import (
"github.com/rs/zerolog"
"golang.org/x/sync/errgroup"
"github.com/cloudflare/cloudflared/client"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/edgediscovery"
"github.com/cloudflare/cloudflared/edgediscovery/allregions"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/fips"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/ingress/origins"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/orchestration"
quicpogs "github.com/cloudflare/cloudflared/quic"
@@ -38,10 +40,8 @@ const (
)
type TunnelConfig struct {
ClientConfig *client.Config
GracePeriod time.Duration
ReplaceExisting bool
OSArch string
ClientID string
CloseConnOnce *sync.Once // Used to close connectedSignal no more than once
EdgeAddrs []string
Region string
@@ -61,10 +61,12 @@ type TunnelConfig struct {
NeedPQ bool
NamedTunnel *connection.TunnelProperties
ProtocolSelector connection.ProtocolSelector
EdgeTLSConfigs map[connection.Protocol]*tls.Config
ICMPRouterServer ingress.ICMPRouterServer
NamedTunnel *connection.TunnelProperties
ProtocolSelector connection.ProtocolSelector
EdgeTLSConfigs map[connection.Protocol]*tls.Config
ICMPRouterServer ingress.ICMPRouterServer
OriginDNSService *origins.DNSResolverService
OriginDialerService *ingress.OriginDialerService
RPCTimeout time.Duration
WriteStreamTimeout time.Duration
@@ -72,22 +74,13 @@ type TunnelConfig struct {
DisableQUICPathMTUDiscovery bool
QUICConnectionLevelFlowControlLimit uint64
QUICStreamLevelFlowControlLimit uint64
FeatureSelector *features.FeatureSelector
}
func (c *TunnelConfig) connectionOptions(originLocalAddr string, numPreviousAttempts uint8) *pogs.ConnectionOptions {
func (c *TunnelConfig) connectionOptions(originLocalAddr string, previousAttempts uint8) *client.ConnectionOptionsSnapshot {
// attempt to parse out origin IP, but don't fail since it's informational field
host, _, _ := net.SplitHostPort(originLocalAddr)
originIP := net.ParseIP(host)
return &pogs.ConnectionOptions{
Client: c.NamedTunnel.Client,
OriginLocalIP: originIP,
ReplaceExisting: c.ReplaceExisting,
CompressionQuality: 0,
NumPreviousAttempts: numPreviousAttempts,
}
return c.ClientConfig.ConnectionOptionsSnapshot(originIP, previousAttempts)
}
func StartTunnelDaemon(
@@ -463,6 +456,8 @@ func (e *EdgeTunnelServer) serveConnection(
case connection.QUIC:
// nolint: gosec
connOptions := e.config.connectionOptions(addr.UDP.String(), uint8(backoff.Retries()))
// nolint: zerologlint
connOptions.LogFields(connLog.Logger().Debug().Uint8(connection.LogFieldConnIndex, connIndex)).Msgf("Tunnel connection options")
return e.serveQUIC(ctx,
addr.UDP.AddrPort(),
connLog,
@@ -479,6 +474,8 @@ func (e *EdgeTunnelServer) serveConnection(
// nolint: gosec
connOptions := e.config.connectionOptions(edgeConn.LocalAddr().String(), uint8(backoff.Retries()))
// nolint: zerologlint
connOptions.LogFields(connLog.Logger().Debug().Uint8(connection.LogFieldConnIndex, connIndex)).Msgf("Tunnel connection options")
if err := e.serveHTTP2(
ctx,
connLog,
@@ -508,11 +505,11 @@ func (e *EdgeTunnelServer) serveHTTP2(
ctx context.Context,
connLog *ConnAwareLogger,
tlsServerConn net.Conn,
connOptions *pogs.ConnectionOptions,
connOptions *client.ConnectionOptionsSnapshot,
controlStreamHandler connection.ControlStreamHandler,
connIndex uint8,
) error {
pqMode := e.config.FeatureSelector.PostQuantumMode()
pqMode := connOptions.FeatureSnapshot.PostQuantum
if pqMode == features.PostQuantumStrict {
return unrecoverableError{errors.New("HTTP/2 transport does not support post-quantum")}
}
@@ -550,19 +547,19 @@ func (e *EdgeTunnelServer) serveQUIC(
ctx context.Context,
edgeAddr netip.AddrPort,
connLogger *ConnAwareLogger,
connOptions *pogs.ConnectionOptions,
connOptions *client.ConnectionOptionsSnapshot,
controlStreamHandler connection.ControlStreamHandler,
connIndex uint8,
) (err error, recoverable bool) {
tlsConfig := e.config.EdgeTLSConfigs[connection.QUIC]
pqMode := e.config.FeatureSelector.PostQuantumMode()
pqMode := connOptions.FeatureSnapshot.PostQuantum
curvePref, err := curvePreference(pqMode, fips.IsFipsEnabled(), tlsConfig.CurvePreferences)
if err != nil {
return err, true
}
connLogger.Logger().Info().Msgf("Using %v as curve preferences", curvePref)
connLogger.Logger().Info().Msgf("Tunnel connection curve preferences: %v", curvePref)
tlsConfig.CurvePreferences = curvePref
@@ -600,12 +597,12 @@ func (e *EdgeTunnelServer) serveQUIC(
if err != nil {
connLogger.ConnAwareLogger().Err(err).Msgf("Failed to dial a quic connection")
e.reportErrorToSentry(err)
e.reportErrorToSentry(err, connOptions.FeatureSnapshot.PostQuantum)
return err, true
}
var datagramSessionManager connection.DatagramSessionHandler
if e.config.FeatureSelector.DatagramVersion() == features.DatagramV3 {
if connOptions.FeatureSnapshot.DatagramVersion == features.DatagramV3 {
datagramSessionManager = connection.NewDatagramV3Connection(
ctx,
conn,
@@ -619,6 +616,7 @@ func (e *EdgeTunnelServer) serveQUIC(
datagramSessionManager = connection.NewDatagramV2Connection(
ctx,
conn,
e.config.OriginDialerService,
e.config.ICMPRouterServer,
connIndex,
e.config.RPCTimeout,
@@ -672,7 +670,7 @@ func (e *EdgeTunnelServer) serveQUIC(
// The reportErrorToSentry is an helper function that handles
// verifies if an error should be reported to Sentry.
func (e *EdgeTunnelServer) reportErrorToSentry(err error) {
func (e *EdgeTunnelServer) reportErrorToSentry(err error, pqMode features.PostQuantumMode) {
dialErr, ok := err.(*connection.EdgeQuicDialError)
if ok {
// The TransportError provides an Unwrap function however
@@ -681,7 +679,7 @@ func (e *EdgeTunnelServer) reportErrorToSentry(err error) {
if ok &&
transportErr.ErrorCode.IsCryptoError() &&
fips.IsFipsEnabled() &&
e.config.FeatureSelector.PostQuantumMode() == features.PostQuantumStrict {
pqMode == features.PostQuantumStrict {
// Only report to Sentry when using FIPS, PQ,
// and the error is a Crypto error reported by
// an EdgeQuicDialError
+1 -1
View File
@@ -96,7 +96,7 @@ func (p *UpdateConfigurationResponse) Unmarshal(s proto.UpdateConfigurationRespo
return err
}
if respErr != "" {
p.Err = fmt.Errorf(respErr)
p.Err = fmt.Errorf("%s", respErr)
}
return nil
}
+2 -2
View File
@@ -127,7 +127,7 @@ func (p *RegisterUdpSessionResponse) Unmarshal(s proto.RegisterUdpSessionRespons
return err
}
if respErr != "" {
p.Err = fmt.Errorf(respErr)
p.Err = fmt.Errorf("%s", respErr)
}
p.Spans, err = s.Spans()
if err != nil {
@@ -164,7 +164,7 @@ func (c SessionManager_PogsClient) RegisterUdpSession(ctx context.Context, sessi
}
p.SetDstPort(dstPort)
p.SetCloseAfterIdleHint(int64(closeAfterIdleHint))
p.SetTraceContext(traceContext)
_ = p.SetTraceContext(traceContext)
return nil
})
result, err := promise.Result().Struct()
+11 -11
View File
@@ -22,7 +22,7 @@ const (
)
func TestConnectRequestData(t *testing.T) {
var tests = []struct {
tests := []struct {
name string
hostname string
connectionType pogs.ConnectionType
@@ -62,7 +62,7 @@ func TestConnectRequestData(t *testing.T) {
}
func TestConnectResponseMeta(t *testing.T) {
var tests = []struct {
tests := []struct {
name string
err error
metadata []pogs.Metadata
@@ -106,7 +106,7 @@ func TestConnectResponseMeta(t *testing.T) {
func TestRegisterUdpSession(t *testing.T) {
unregisterMessage := "closed by eyeball"
var tests = []struct {
tests := []struct {
name string
sessionRPCServer mockSessionRPCServer
}{
@@ -140,29 +140,29 @@ func TestRegisterUdpSession(t *testing.T) {
sessionRegisteredChan := make(chan struct{})
go func() {
ss := NewCloudflaredServer(nil, test.sessionRPCServer, nil, 10*time.Second)
err := ss.Serve(context.Background(), serverStream)
err := ss.Serve(t.Context(), serverStream)
assert.NoError(t, err)
serverStream.Close()
close(sessionRegisteredChan)
}()
rpcClientStream, err := NewCloudflaredClient(context.Background(), clientStream, 5*time.Second)
rpcClientStream, err := NewCloudflaredClient(t.Context(), clientStream, 5*time.Second)
require.NoError(t, err)
reg, err := rpcClientStream.RegisterUdpSession(context.Background(), test.sessionRPCServer.sessionID, test.sessionRPCServer.dstIP, test.sessionRPCServer.dstPort, testCloseIdleAfterHint, test.sessionRPCServer.traceContext)
reg, err := rpcClientStream.RegisterUdpSession(t.Context(), test.sessionRPCServer.sessionID, test.sessionRPCServer.dstIP, test.sessionRPCServer.dstPort, testCloseIdleAfterHint, test.sessionRPCServer.traceContext)
require.NoError(t, err)
require.NoError(t, reg.Err)
// Different sessionID, the RPC server should reject the registration
reg, err = rpcClientStream.RegisterUdpSession(context.Background(), uuid.New(), test.sessionRPCServer.dstIP, test.sessionRPCServer.dstPort, testCloseIdleAfterHint, test.sessionRPCServer.traceContext)
reg, err = rpcClientStream.RegisterUdpSession(t.Context(), uuid.New(), test.sessionRPCServer.dstIP, test.sessionRPCServer.dstPort, testCloseIdleAfterHint, test.sessionRPCServer.traceContext)
require.NoError(t, err)
require.Error(t, reg.Err)
require.NoError(t, rpcClientStream.UnregisterUdpSession(context.Background(), test.sessionRPCServer.sessionID, unregisterMessage))
require.NoError(t, rpcClientStream.UnregisterUdpSession(t.Context(), test.sessionRPCServer.sessionID, unregisterMessage))
// Different sessionID, the RPC server should reject the unregistration
require.Error(t, rpcClientStream.UnregisterUdpSession(context.Background(), uuid.New(), unregisterMessage))
require.Error(t, rpcClientStream.UnregisterUdpSession(t.Context(), uuid.New(), unregisterMessage))
rpcClientStream.Close()
<-sessionRegisteredChan
@@ -185,14 +185,14 @@ func TestManageConfiguration(t *testing.T) {
updatedChan := make(chan struct{})
go func() {
server := NewCloudflaredServer(nil, nil, configRPCServer, 10*time.Second)
err := server.Serve(context.Background(), serverStream)
err := server.Serve(t.Context(), serverStream)
assert.NoError(t, err)
serverStream.Close()
close(updatedChan)
}()
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
ctx, cancel := context.WithTimeout(t.Context(), time.Second)
defer cancel()
rpcClientStream, err := NewCloudflaredClient(ctx, clientStream, 5*time.Second)
require.NoError(t, err)
+2
View File
@@ -70,3 +70,5 @@ benchstat <(go test -benchtime 500ms -count 15 -bench 'Sum64$')
- [VictoriaMetrics](https://github.com/VictoriaMetrics/VictoriaMetrics)
- [FreeCache](https://github.com/coocood/freecache)
- [FastCache](https://github.com/VictoriaMetrics/fastcache)
- [Ristretto](https://github.com/dgraph-io/ristretto)
- [Badger](https://github.com/dgraph-io/badger)
+22 -7
View File
@@ -19,10 +19,13 @@ const (
// Store the primes in an array as well.
//
// The consts are used when possible in Go code to avoid MOVs but we need a
// contiguous array of the assembly code.
// contiguous array for the assembly code.
var primes = [...]uint64{prime1, prime2, prime3, prime4, prime5}
// Digest implements hash.Hash64.
//
// Note that a zero-valued Digest is not ready to receive writes.
// Call Reset or create a Digest using New before calling other methods.
type Digest struct {
v1 uint64
v2 uint64
@@ -33,19 +36,31 @@ type Digest struct {
n int // how much of mem is used
}
// New creates a new Digest that computes the 64-bit xxHash algorithm.
// New creates a new Digest with a zero seed.
func New() *Digest {
return NewWithSeed(0)
}
// NewWithSeed creates a new Digest with the given seed.
func NewWithSeed(seed uint64) *Digest {
var d Digest
d.Reset()
d.ResetWithSeed(seed)
return &d
}
// Reset clears the Digest's state so that it can be reused.
// It uses a seed value of zero.
func (d *Digest) Reset() {
d.v1 = primes[0] + prime2
d.v2 = prime2
d.v3 = 0
d.v4 = -primes[0]
d.ResetWithSeed(0)
}
// ResetWithSeed clears the Digest's state so that it can be reused.
// It uses the given seed to initialize the state.
func (d *Digest) ResetWithSeed(seed uint64) {
d.v1 = seed + prime1 + prime2
d.v2 = seed + prime2
d.v3 = seed
d.v4 = seed - prime1
d.total = 0
d.n = 0
}
+1 -1
View File
@@ -6,7 +6,7 @@
package xxhash
// Sum64 computes the 64-bit xxHash digest of b.
// Sum64 computes the 64-bit xxHash digest of b with a zero seed.
//
//go:noescape
func Sum64(b []byte) uint64
+1 -1
View File
@@ -3,7 +3,7 @@
package xxhash
// Sum64 computes the 64-bit xxHash digest of b.
// Sum64 computes the 64-bit xxHash digest of b with a zero seed.
func Sum64(b []byte) uint64 {
// A simpler version would be
// d := New()
+1 -1
View File
@@ -5,7 +5,7 @@
package xxhash
// Sum64String computes the 64-bit xxHash digest of s.
// Sum64String computes the 64-bit xxHash digest of s with a zero seed.
func Sum64String(s string) uint64 {
return Sum64([]byte(s))
}
+1 -1
View File
@@ -33,7 +33,7 @@ import (
//
// See https://github.com/golang/go/issues/42739 for discussion.
// Sum64String computes the 64-bit xxHash digest of s.
// Sum64String computes the 64-bit xxHash digest of s with a zero seed.
// It may be faster than Sum64([]byte(s)) by avoiding a copy.
func Sum64String(s string) uint64 {
b := *(*[]byte)(unsafe.Pointer(&sliceHeader{s, len(s)}))
+4
View File
@@ -1,5 +1,9 @@
THIS IS A FORK OF CADDY v1 - EVERYTHING IS STRIPPED EXCEPT THE PIECES NEEDED IN COREDNS.
Issues are not enabled in this repository. Please raise any issues in coredns/coredns.
---
Caddy is a **production-ready** open-source web server that is fast, easy to use, and makes you more productive.
+18 -7
View File
@@ -1,5 +1,7 @@
// Copyright 2015 Light Code Labs, LLC
//
// Copyright 2024 MWS
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
@@ -218,22 +220,22 @@ func (i *Instance) Restart(newCaddyfile Input) (*Instance, error) {
}
// Add file descriptors of all the sockets that are capable of it
restartFds := make(map[string]restartTriple)
restartFds := make(map[string][]restartTriple)
for _, s := range i.servers {
gs, srvOk := s.server.(GracefulServer)
ln, lnOk := s.listener.(Listener)
pc, pcOk := s.packet.(PacketConn)
if srvOk {
if lnOk && pcOk {
restartFds[gs.Address()] = restartTriple{server: gs, listener: ln, packet: pc}
restartFds[gs.Address()] = append(restartFds[gs.Address()], restartTriple{server: gs, listener: ln, packet: pc})
continue
}
if lnOk {
restartFds[gs.Address()] = restartTriple{server: gs, listener: ln}
restartFds[gs.Address()] = append(restartFds[gs.Address()], restartTriple{server: gs, listener: ln})
continue
}
if pcOk {
restartFds[gs.Address()] = restartTriple{server: gs, packet: pc}
restartFds[gs.Address()] = append(restartFds[gs.Address()], restartTriple{server: gs, packet: pc})
continue
}
}
@@ -484,7 +486,7 @@ func Start(cdyfile Input) (*Instance, error) {
return inst, nil
}
func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]restartTriple) error {
func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string][]restartTriple) error {
// save this instance in the list now so that
// plugins can access it if need be, for example
// the caddytls package, so it can perform cert
@@ -684,7 +686,7 @@ func executeDirectives(inst *Instance, filename string,
return nil
}
func startServers(serverList []Server, inst *Instance, restartFds map[string]restartTriple) error {
func startServers(serverList []Server, inst *Instance, restartFds map[string][]restartTriple) error {
errChan := make(chan error, len(serverList))
// used for signaling to error logging goroutine to terminate
@@ -734,7 +736,16 @@ func startServers(serverList []Server, inst *Instance, restartFds map[string]res
// reuse the listener for a graceful restart.
if gs, ok := s.(GracefulServer); ok && restartFds != nil {
addr := gs.Address()
if old, ok := restartFds[addr]; ok {
// Multiple servers may use the same addr (SO_REUSEPORT option set), so it's important to ensure
// that we don't reuse the same listener/packetconn.
// We'll create new listeners in case there are no more available triples for the same address.
if triples, ok := restartFds[addr]; ok && len(triples) > 0 {
// Take first available triple
old := triples[0]
// Remove reused triple from restartFds
triples[0] = triples[len(triples)-1]
restartFds[addr] = triples[:len(triples)-1]
// listener
if old.listener != nil {
file, err := old.listener.File()
+6 -26
View File
@@ -56,33 +56,13 @@ var (
func DescribePlugins() string {
pl := ListPlugins()
str := "Server types:\n"
for _, name := range pl["server_types"] {
str += " " + name + "\n"
}
str += "\nCaddyfile loaders:\n"
for _, name := range pl["caddyfile_loaders"] {
str += " " + name + "\n"
}
if len(pl["event_hooks"]) > 0 {
str += "\nEvent hook plugins:\n"
for _, name := range pl["event_hooks"] {
str += " hook." + name + "\n"
}
}
if len(pl["clustering"]) > 0 {
str += "\nClustering plugins:\n"
for _, name := range pl["clustering"] {
str += " " + name + "\n"
}
}
str += "\nOther plugins:\n"
str := ""
for _, name := range pl["others"] {
str += " " + name + "\n"
if len(name) > 3 {
str += name[4:] + "\n" // drop dns. prefix caddy adds
} else {
str += name + "\n"
}
}
return str
+12
View File
@@ -24,6 +24,10 @@ type Config struct {
// The port to listen on.
Port string
// The number of servers that will listen on one port.
// By default, one server will be running.
NumSockets int
// Root points to a base directory we find user defined "things".
// First consumer is the file plugin to looks for zone files in this place.
Root string
@@ -54,6 +58,14 @@ type Config struct {
// TLSConfig when listening for encrypted connections (gRPC, DNS-over-TLS).
TLSConfig *tls.Config
// MaxQUICStreams defines the maximum number of concurrent QUIC streams for a QUIC server.
// This is nil if not specified, allowing for a default to be used.
MaxQUICStreams *int
// MaxQUICWorkerPoolSize defines the size of the worker pool for processing QUIC streams.
// This is nil if not specified, allowing for a default to be used.
MaxQUICWorkerPoolSize *int
// Timeouts for TCP, TLS and HTTPS servers.
ReadTimeout time.Duration
WriteTimeout time.Duration
+90 -55
View File
@@ -134,69 +134,23 @@ func (h *dnsContext) InspectServerBlocks(sourceFile string, serverBlocks []caddy
// MakeServers uses the newly-created siteConfigs to create and return a list of server instances.
func (h *dnsContext) MakeServers() ([]caddy.Server, error) {
// Copy the Plugin, ListenHosts and Debug from first config in the block
// to all other config in the same block . Doing this results in zones
// sharing the same plugin instances and settings as other zones in
// the same block.
for _, c := range h.configs {
c.Plugin = c.firstConfigInBlock.Plugin
c.ListenHosts = c.firstConfigInBlock.ListenHosts
c.Debug = c.firstConfigInBlock.Debug
c.Stacktrace = c.firstConfigInBlock.Stacktrace
// Fork TLSConfig for each encrypted connection
c.TLSConfig = c.firstConfigInBlock.TLSConfig.Clone()
c.ReadTimeout = c.firstConfigInBlock.ReadTimeout
c.WriteTimeout = c.firstConfigInBlock.WriteTimeout
c.IdleTimeout = c.firstConfigInBlock.IdleTimeout
c.TsigSecret = c.firstConfigInBlock.TsigSecret
}
// Copy parameters from first config in the block to all other config in the same block
propagateConfigParams(h.configs)
// we must map (group) each config to a bind address
groups, err := groupConfigsByListenAddr(h.configs)
if err != nil {
return nil, err
}
// then we create a server for each group
var servers []caddy.Server
for addr, group := range groups {
// switch on addr
switch tr, _ := parse.Transport(addr); tr {
case transport.DNS:
s, err := NewServer(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
case transport.TLS:
s, err := NewServerTLS(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
case transport.QUIC:
s, err := NewServerQUIC(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
case transport.GRPC:
s, err := NewServergRPC(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
case transport.HTTPS:
s, err := NewServerHTTPS(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
serversForGroup, err := makeServersForGroup(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, serversForGroup...)
}
// For each server config, check for View Filter plugins
@@ -263,8 +217,11 @@ func (c *Config) Handlers() []plugin.Handler {
return nil
}
hs := make([]plugin.Handler, 0, len(c.registry))
for k := range c.registry {
hs = append(hs, c.registry[k])
for _, k := range Directives {
registry := c.Handler(k)
if registry != nil {
hs = append(hs, registry)
}
}
return hs
}
@@ -296,6 +253,27 @@ func (h *dnsContext) validateZonesAndListeningAddresses() error {
return nil
}
// propagateConfigParams copies the necessary parameters from first config in the block
// to all other config in the same block. Doing this results in zones
// sharing the same plugin instances and settings as other zones in
// the same block.
func propagateConfigParams(configs []*Config) {
for _, c := range configs {
c.Plugin = c.firstConfigInBlock.Plugin
c.ListenHosts = c.firstConfigInBlock.ListenHosts
c.Debug = c.firstConfigInBlock.Debug
c.Stacktrace = c.firstConfigInBlock.Stacktrace
c.NumSockets = c.firstConfigInBlock.NumSockets
// Fork TLSConfig for each encrypted connection
c.TLSConfig = c.firstConfigInBlock.TLSConfig.Clone()
c.ReadTimeout = c.firstConfigInBlock.ReadTimeout
c.WriteTimeout = c.firstConfigInBlock.WriteTimeout
c.IdleTimeout = c.firstConfigInBlock.IdleTimeout
c.TsigSecret = c.firstConfigInBlock.TsigSecret
}
}
// groupConfigsByListenAddr groups site configs by their listen
// (bind) address, so sites that use the same listener can be served
// on the same server instance. The return value maps the listen
@@ -317,6 +295,63 @@ func groupConfigsByListenAddr(configs []*Config) (map[string][]*Config, error) {
return groups, nil
}
// makeServersForGroup creates servers for a specific transport and group.
// It creates as many servers as specified in the NumSockets configuration.
// If the NumSockets param is not specified, one server is created by default.
func makeServersForGroup(addr string, group []*Config) ([]caddy.Server, error) {
// that is impossible, but better to check
if len(group) == 0 {
return nil, fmt.Errorf("no configs for group defined")
}
// create one server by default if no NumSockets specified
numSockets := 1
if group[0].NumSockets > 0 {
numSockets = group[0].NumSockets
}
var servers []caddy.Server
for range numSockets {
// switch on addr
switch tr, _ := parse.Transport(addr); tr {
case transport.DNS:
s, err := NewServer(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
case transport.TLS:
s, err := NewServerTLS(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
case transport.QUIC:
s, err := NewServerQUIC(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
case transport.GRPC:
s, err := NewServergRPC(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
case transport.HTTPS:
s, err := NewServerHTTPS(addr, group)
if err != nil {
return nil, err
}
servers = append(servers, s)
}
}
return servers, nil
}
// DefaultPort is the default port.
const DefaultPort = transport.Port
+1 -1
View File
@@ -123,7 +123,7 @@ func (s *ServergRPC) Stop() (err error) {
// back to the client as a protobuf.
func (s *ServergRPC) Query(ctx context.Context, in *pb.DnsPacket) (*pb.DnsPacket, error) {
msg := new(dns.Msg)
err := msg.Unpack(in.Msg)
err := msg.Unpack(in.GetMsg())
if err != nil {
return nil, err
}
+1 -1
View File
@@ -188,7 +188,7 @@ func (s *ServerHTTPS) ServeHTTP(w http.ResponseWriter, r *http.Request) {
age := dnsutil.MinimalTTL(dw.Msg, mt)
w.Header().Set("Content-Type", doh.MimeType)
w.Header().Set("Cache-Control", fmt.Sprintf("max-age=%f", age.Seconds()))
w.Header().Set("Cache-Control", fmt.Sprintf("max-age=%d", uint32(age.Seconds())))
w.Header().Set("Content-Length", strconv.Itoa(len(buf)))
w.WriteHeader(http.StatusOK)
s.countResponse(http.StatusOK)
+41 -11
View File
@@ -7,7 +7,6 @@ import (
"errors"
"fmt"
"io"
"math"
"net"
"github.com/coredns/coredns/plugin/metrics/vars"
@@ -32,15 +31,26 @@ const (
// DoQCodeProtocolError signals that the DoQ implementation encountered
// a protocol error and is forcibly aborting the connection.
DoQCodeProtocolError quic.ApplicationErrorCode = 2
// DefaultMaxQUICStreams is the default maximum number of concurrent QUIC streams
// on a per-connection basis. RFC 9250 (DNS-over-QUIC) does not require a high
// concurrent-stream limit; normal stub or recursive resolvers open only a handful
// of streams in parallel. This default (256) is a safe upper bound.
DefaultMaxQUICStreams = 256
// DefaultQUICStreamWorkers is the default number of workers for processing QUIC streams.
DefaultQUICStreamWorkers = 1024
)
// ServerQUIC represents an instance of a DNS-over-QUIC server.
type ServerQUIC struct {
*Server
listenAddr net.Addr
tlsConfig *tls.Config
quicConfig *quic.Config
quicListener *quic.Listener
listenAddr net.Addr
tlsConfig *tls.Config
quicConfig *quic.Config
quicListener *quic.Listener
maxStreams int
streamProcessPool chan struct{}
}
// NewServerQUIC returns a new CoreDNS QUIC server and compiles all plugin in to it.
@@ -63,16 +73,31 @@ func NewServerQUIC(addr string, group []*Config) (*ServerQUIC, error) {
tlsConfig.NextProtos = []string{"doq"}
}
var quicConfig *quic.Config
quicConfig = &quic.Config{
maxStreams := DefaultMaxQUICStreams
if len(group) > 0 && group[0] != nil && group[0].MaxQUICStreams != nil {
maxStreams = *group[0].MaxQUICStreams
}
streamProcessPoolSize := DefaultQUICStreamWorkers
if len(group) > 0 && group[0] != nil && group[0].MaxQUICWorkerPoolSize != nil {
streamProcessPoolSize = *group[0].MaxQUICWorkerPoolSize
}
var quicConfig = &quic.Config{
MaxIdleTimeout: s.idleTimeout,
MaxIncomingStreams: math.MaxUint16,
MaxIncomingUniStreams: math.MaxUint16,
MaxIncomingStreams: int64(maxStreams),
MaxIncomingUniStreams: int64(maxStreams),
// Enable 0-RTT by default for all connections on the server-side.
Allow0RTT: true,
}
return &ServerQUIC{Server: s, tlsConfig: tlsConfig, quicConfig: quicConfig}, nil
return &ServerQUIC{
Server: s,
tlsConfig: tlsConfig,
quicConfig: quicConfig,
maxStreams: maxStreams,
streamProcessPool: make(chan struct{}, streamProcessPoolSize),
}, nil
}
// ServePacket implements caddy.UDPServer interface.
@@ -120,7 +145,12 @@ func (s *ServerQUIC) serveQUICConnection(conn quic.Connection) {
return
}
go s.serveQUICStream(stream, conn)
// Use a bounded worker pool
s.streamProcessPool <- struct{}{} // Acquire a worker slot, may block
go func(st quic.Stream, cn quic.Connection) {
defer func() { <-s.streamProcessPool }() // Release worker slot
s.serveQUICStream(st, cn)
}(stream, conn)
}
}
+2
View File
@@ -15,7 +15,9 @@ var Directives = []string{
"geoip",
"cancel",
"tls",
"quic",
"timeouts",
"multisocket",
"reload",
"nsid",
"bufsize",
+8 -2
View File
@@ -12,6 +12,8 @@ import (
"github.com/coredns/caddy"
"github.com/coredns/coredns/core/dnsserver"
"go.uber.org/automaxprocs/maxprocs"
)
func init() {
@@ -31,7 +33,7 @@ func init() {
flag.StringVar(&dnsserver.Port, serverType+".port", dnsserver.DefaultPort, "Default port")
flag.StringVar(&dnsserver.Port, "p", dnsserver.DefaultPort, "Default port")
caddy.AppName = coreName
caddy.AppName = CoreName
caddy.AppVersion = CoreVersion
}
@@ -56,6 +58,11 @@ func Run() {
os.Exit(0)
}
_, err := maxprocs.Set(maxprocs.Logger(log.Printf))
if err != nil {
log.Println("[WARNING] Failed to set GOMAXPROCS:", err)
}
// Get Corefile input
corefile, err := caddy.LoadCaddyfile(serverType)
if err != nil {
@@ -176,7 +183,6 @@ var (
// Build information obtained with the help of -ldflags
var (
// nolint
appVersion = "(untracked dev build)" // inferred at startup
devBuild = true // inferred at startup
+2 -2
View File
@@ -2,7 +2,7 @@ package coremain
// Various CoreDNS constants.
const (
CoreVersion = "1.11.3"
coreName = "CoreDNS"
CoreVersion = "1.12.2"
CoreName = "CoreDNS"
serverType = "dns"
)
+10 -8
View File
@@ -13,6 +13,8 @@ import (
"github.com/miekg/dns"
)
const maxCnameChainLength = 10
// A returns A records from Backend or an error.
func A(ctx context.Context, b ServiceBackend, zone string, state request.Request, previousRecords []dns.RR, opt Options) (records []dns.RR, truncated bool, err error) {
services, err := checkForApex(ctx, b, zone, state, opt)
@@ -34,7 +36,7 @@ func A(ctx context.Context, b ServiceBackend, zone string, state request.Request
}
newRecord := serv.NewCNAME(state.QName(), serv.Host)
if len(previousRecords) > 7 {
if len(previousRecords) > maxCnameChainLength {
// don't add it, and just continue
continue
}
@@ -108,7 +110,7 @@ func AAAA(ctx context.Context, b ServiceBackend, zone string, state request.Requ
}
newRecord := serv.NewCNAME(state.QName(), serv.Host)
if len(previousRecords) > 7 {
if len(previousRecords) > maxCnameChainLength {
// don't add it, and just continue
continue
}
@@ -361,7 +363,7 @@ func TXT(ctx context.Context, b ServiceBackend, zone string, state request.Reque
}
newRecord := serv.NewCNAME(state.QName(), serv.Host)
if len(previousRecords) > 7 {
if len(previousRecords) > maxCnameChainLength {
// don't add it, and just continue
continue
}
@@ -427,20 +429,20 @@ func PTR(ctx context.Context, b ServiceBackend, zone string, state request.Reque
return records, nil
}
// NS returns NS records from the backend
// NS returns NS records from the backend
func NS(ctx context.Context, b ServiceBackend, zone string, state request.Request, opt Options) (records, extra []dns.RR, err error) {
// NS record for this zone live in a special place, ns.dns.<zone>. Fake our lookup.
// only a tad bit fishy...
// NS record for this zone lives in a special place, ns.dns.<zone>. Fake our lookup.
// Only a tad bit fishy...
old := state.QName()
state.Clear()
state.Req.Question[0].Name = dnsutil.Join("ns.dns.", zone)
services, err := b.Services(ctx, state, false, opt)
// reset the query name to the original
state.Req.Question[0].Name = old
if err != nil {
return nil, nil, err
}
// ... and reset
state.Req.Question[0].Name = old
seen := map[string]bool{}
+4 -3
View File
@@ -189,11 +189,12 @@ func (w *ResponseWriter) WriteMsg(res *dns.Msg) error {
msgTTL := dnsutil.MinimalTTL(res, mt)
var duration time.Duration
if mt == response.NameError || mt == response.NoData {
switch mt {
case response.NameError, response.NoData:
duration = computeTTL(msgTTL, w.minnttl, w.nttl)
} else if mt == response.ServerError {
case response.ServerError:
duration = w.failttl
} else {
default:
duration = computeTTL(msgTTL, w.minpttl, w.pttl)
}
+4 -5
View File
@@ -34,14 +34,13 @@ func (c *Cache) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg)
// in which upstream doesn't support DNSSEC, the two cache items will effectively be the same. Regardless, any
// DNSSEC RRs in the response are written to cache with the response.
ttl := 0
i := c.getIgnoreTTL(now, state, server)
if i == nil {
crr := &ResponseWriter{ResponseWriter: w, Cache: c, state: state, server: server, do: do, ad: ad, cd: cd,
nexcept: c.nexcept, pexcept: c.pexcept, wildcardFunc: wildcardFunc(ctx)}
return c.doRefresh(ctx, state, crr)
}
ttl = i.ttl(now)
ttl := i.ttl(now)
if ttl < 0 {
// serve stale behavior
if c.verifyStale {
@@ -100,7 +99,7 @@ func (c *Cache) doPrefetch(ctx context.Context, state request.Request, cw *Respo
// that we've gathered sofar. See we copy the frequencies info back
// into the new item that was stored in the cache.
if i1 := c.exists(state); i1 != nil {
i1.Freq.Reset(now, i.Freq.Hits())
i1.Reset(now, i.Hits())
}
}
@@ -112,9 +111,9 @@ func (c *Cache) shouldPrefetch(i *item, now time.Time) bool {
if c.prefetch <= 0 {
return false
}
i.Freq.Update(c.duration, now)
i.Update(c.duration, now)
threshold := int(math.Ceil(float64(c.percentage) / 100 * float64(i.origTTL)))
return i.Freq.Hits() >= c.prefetch && i.ttl(now) <= threshold
return i.Hits() >= c.prefetch && i.ttl(now) <= threshold
}
// Name implements the Handler interface.
+3 -4
View File
@@ -149,12 +149,11 @@ func split255(s string) []string {
sx := []string{}
p, i := 0, 255
for {
if i <= len(s) {
sx = append(sx, s[p:i])
} else {
if i > len(s) {
sx = append(sx, s[p:])
break
}
sx = append(sx, s[p:i])
p, i = p+255, i+255
}
@@ -164,7 +163,7 @@ func split255(s string) []string {
// targetStrip strips "targetstrip" labels from the left side of the fully qualified name.
func targetStrip(name string, targetStrip int) string {
offset, end := 0, false
for i := 0; i < targetStrip; i++ {
for range targetStrip {
offset, end = dns.NextLabel(name, offset)
}
if end {

Some files were not shown because too many files have changed in this diff Show More