Compare commits

...

58 Commits

Author SHA1 Message Date
Chung-Ting Huang 8ee1faf317 Release 2019.11.2 2019-11-19 15:16:15 -06:00
Chung-Ting Huang c5bacf4d95 TUN-2563: Exposes config_version metrics 2019-11-19 20:42:41 +00:00
Nick Vollmar 1ba5abfdb3 Release 2019.11.1 2019-11-19 19:56:25 +00:00
Adam Chalmers 7173da9359 TUN-2567: AuthOutcome can be turned back into AuthResponse 2019-11-19 04:34:00 +00:00
Nick Vollmar 0676923d24 TUN-2551: TunnelRPC definitions for ReconnectTunnel flow 2019-11-18 14:26:58 -06:00
Adam Chalmers ca7fbf43da TUN-2547: TunnelRPC definitions for Authenticate flow 2019-11-18 16:39:01 +00:00
Nick Vollmar 6ea9b5c3ff TUN-2490: respect original representation of HTTP request path 2019-11-14 22:56:07 -06:00
Nick Vollmar 1d96bccc04 TUN-2178: public API to create new h2mux.MuxedStreamRequest 2019-11-13 14:04:19 -06:00
Adam Chalmers c5af83af66 TUN-2053: Add a /healthcheck endpoint to the metrics server 2019-11-12 16:55:40 -06:00
Ashcon Partovi 759cd019be Add db-connect, a SQL over HTTPS server 2019-11-12 20:34:39 +00:00
Chung-Ting Huang 13bf65ce4e TUN-2506: Expose active streams metrics 2019-11-07 14:09:31 -06:00
Chung-Ting Huang 3a2e12818f Release 2019.11.0 2019-11-07 10:11:56 -06:00
Nick Vollmar e14ec1a1fb TUN-2505: Terminate stream on receipt of RST_STREAM; MuxedStream.CloseWrite() should terminate the MuxedStream.Write() loop 2019-11-06 21:24:18 +00:00
Chung-Ting Huang 3a9a0a0d75 TUN-2489: Delete stream from activestreammap when read and write are both closed 2019-11-05 11:06:11 -06:00
Areg Harutyunyan 068b148e05 TUN-2500: Don't send client registration errors to Sentry 2019-11-05 12:29:11 +00:00
Nick Vollmar e5335b6c1b TUN-2502: Switch to go modules 2019-11-04 15:05:02 -06:00
Chung-Ting Huang fa841fc89a TUN-2451: Log inavlid path 2019-10-30 14:07:28 -05:00
Michael Borkenstein e9a8c9850f AUTH-2177: Reads and writes error streams 2019-10-25 19:44:14 +00:00
Chung-Ting Huang a52f47c9a9 TUN-2460: Configure according to the ClientConfig recevied from a successful Connect 2019-10-24 14:53:14 +00:00
Michael Borkenstein ad9559c66a AUTH-2173: Prepends access login url with scheme if one doesnt exist 2019-10-23 20:35:12 +00:00
Chung-Ting Huang 7133eceb9b TUN-2425: Enable cloudflared to serve multiple Hello World servers by having each of them create its own ServeMux 2019-10-22 10:09:59 -05:00
Areg Harutyunyan e0c2758b64 Release 2019.10.4 2019-10-21 14:48:22 -05:00
Areg Harutyunyan 5464408ea7 TUN-2450: Remove Brew publishing formula 2019-10-21 13:44:49 -05:00
Areg Harutyunyan 802e538c42 Release 2019.10.3 2019-10-21 10:20:34 -05:00
Areg Harutyunyan 79065514ff Merge remote-tracking branch 'upstream/master' 2019-10-18 15:33:27 -05:00
Michael Borkenstein 51acf2ef08 Release 2019.10.2 2019-10-18 13:11:07 -05:00
Felix Bünemann 1f6a330098 Fix #129: Excessive memory usage streaming large files (#142)
This drops the default size auf the h2mux write buffer from 512 MB to 1 MB.
This massively reduces memory usage, since each stream has its own buffer.
2019-10-17 17:15:51 -05:00
Michael Borkenstein 28cc1c65af AUTH-2167: Adds CLI option for host key directory 2019-10-17 16:31:43 -05:00
Michael Borkenstein 6322c5029d Release 2019.10.1 2019-10-17 13:17:42 -05:00
Michael Borkenstein 4bb5b97518 Adds variable to fix windows build 2019-10-17 18:15:59 +00:00
Michael Borkenstein 99daa76715 Release 2019.10.0 2019-10-17 10:58:33 -05:00
Michael Borkenstein babe684141 AUTH-2135: Adds support for IPv6 and tests 2019-10-16 16:06:07 -05:00
Michael Borkenstein 8b6e3bc1d1 AUTH-2159: Moves shutdownC close into error handling
AUTH-2161: Lowers size of preamble length
AUTH-2160: Fixes url parsing logic
2019-10-16 11:41:51 -05:00
Michael Borkenstein 95704b11fb AUTH-2114: Uses short lived cert auth for outgoing client connection 2019-10-15 14:35:15 -05:00
Nick Vollmar 4d2583edf5 TUN-2344: log more details: http2.Framer.ErrorDetail() if available, connectionID 2019-10-15 10:59:24 -05:00
Michael Borkenstein a4b3ee5959 AUTH-2105: Dont require --destination arg 2019-10-11 12:26:23 -05:00
Michael Borkenstein 91d9dca34e AUTH-2105: Adds support for local forwarding. Refactor auditlogger creation.
AUTH-2088: Adds dynamic destination routing
2019-10-10 15:25:03 -05:00
Michael Borkenstein dbde3870da AUTH-2089: Revise ssh server to function as a proxy 2019-10-07 13:04:04 -05:00
Chung-Ting Huang b3bcce97da Release 2019.9.2 2019-09-26 14:07:22 -05:00
Areg Harutyunyan a99fac1e31 TUN-2355: Roll back TUN-2276
This reverts commit 02f0ed951f.
2019-09-26 14:02:01 -05:00
Michael Borkenstein 71d66ae7ee Release 2019.9.1 2019-09-26 10:18:28 -05:00
Michael Borkenstein 133e6fdc88 AUTH-2077: Quotes open browser command in windows 2019-09-24 18:27:37 +00:00
Michael Borkenstein 1d5cc45ac7 AUTH-2055: Verifies token at edge on access login 2019-09-24 18:22:33 +00:00
Nick Vollmar a412f629c2 TUN-2334: remove tlsConfig.ServerName special case 2019-09-24 15:00:55 +00:00
Michael Borkenstein 979e5be8ab AUTH-2067: Log commands correctly 2019-09-23 20:42:41 +00:00
Michael Borkenstein 2789d0cf36 AUTH-2052: Adds tests for SSH server 2019-09-23 09:19:43 -05:00
Chung-Ting Huang 5bcb2da0fe TUN-2309: Split ConnectResult into ConnectError and ConnectSuccess, each implementing its own capnp serialization logic 2019-09-19 13:47:41 -05:00
Adam Chalmers 4f23da2a6d TUN-2315: Replace Scope with IntentLabel 2019-09-18 15:11:46 -05:00
Chung-Ting Huang fe032843f3 TUN-2307: Capnp is the only serialization format used in tunnelpogs 2019-09-17 16:29:50 +00:00
Michael Borkenstein ff795a7beb AUTH-2056: Writes stderr to its own stream for non-pty connections 2019-09-16 14:43:05 -05:00
Michael Borkenstein 40d9370bb6 AUTH-2050: Adds time.sleep to temporarily avoid hitting tunnel muxer dealock issue 2019-09-12 14:10:34 +00:00
Areg Harutyunyan 02f0ed951f TUN-2276: Path encoding broken 2019-09-11 19:28:23 +00:00
Michael Borkenstein c2a71c5a51 AUTH-2037: Adds support for ssh port forwarding 2019-09-11 10:41:09 -05:00
Dalton 945bf76897 AUTH-1942 added event log to ssh server 2019-09-10 10:16:32 -05:00
Michael Borkenstein d3b254f9ae AUTH-2036: Refactor user retrieval, shutdown after ssh server stops, add custom version string 2019-09-09 17:31:23 +00:00
Dalton ee588eeeaa AUTH-1943 hooked up uploader to logger, added timestamp to session logs, add tests 2019-09-06 15:57:32 -05:00
Adam Chalmers dd521aba29 TUN-2280: Revert "TUN-2260: add name/group to CapnpConnectParameters, remove Scope"
This reverts commit 817c3be9da5465043c2a2fda6c48f7ada760682e.
2019-09-06 15:59:32 +00:00
Adam Chalmers a06390a078 TUN-2201: change SRV records used by cloudflared
This changes cloudflarewarp.com to argotunnel.com and _warp to
_origintunneld. We've changed which zone we host the SRV records
for Argo Tunnel on.
2019-09-06 15:01:58 +00:00
1214 changed files with 426324 additions and 130168 deletions
+4 -1
View File
@@ -9,5 +9,8 @@ guide/public
\#*\#
cscope.*
cloudflared
cloudflared.exe
!cmd/cloudflared/
.DS_Store
.DS_Store
*-session.log
ssh_server_tests/.env
Generated
-709
View File
@@ -1,709 +0,0 @@
# This file is autogenerated, do not edit; changes may be undone by the next 'dep ensure'.
[[projects]]
digest = "1:9f3b30d9f8e0d7040f729b82dcbc8f0dead820a133b3147ce355fc451f32d761"
name = "github.com/BurntSushi/toml"
packages = ["."]
pruneopts = "UT"
revision = "3012a1dbe2e4bd1391d42b32f0577cb7bbc7f005"
version = "v0.3.1"
[[projects]]
branch = "master"
digest = "1:1a200e7e73293b75eb8e5c93d023b5472663432da0b663e1532624fcfede9ca8"
name = "github.com/anmitsu/go-shlex"
packages = ["."]
pruneopts = "UT"
revision = "648efa622239a2f6ff949fed78ee37b48d499ba4"
[[projects]]
digest = "1:4a296b8f487dab7ea37c4558d6e84be8103eb6bd297202e870266ff99ebd769d"
name = "github.com/aws/aws-sdk-go"
packages = [
"aws",
"aws/awserr",
"aws/awsutil",
"aws/client",
"aws/client/metadata",
"aws/corehandlers",
"aws/credentials",
"aws/credentials/ec2rolecreds",
"aws/credentials/endpointcreds",
"aws/credentials/processcreds",
"aws/credentials/stscreds",
"aws/csm",
"aws/defaults",
"aws/ec2metadata",
"aws/endpoints",
"aws/request",
"aws/session",
"aws/signer/v4",
"internal/ini",
"internal/s3err",
"internal/sdkio",
"internal/sdkrand",
"internal/sdkuri",
"internal/shareddefaults",
"private/protocol",
"private/protocol/eventstream",
"private/protocol/eventstream/eventstreamapi",
"private/protocol/json/jsonutil",
"private/protocol/query",
"private/protocol/query/queryutil",
"private/protocol/rest",
"private/protocol/restxml",
"private/protocol/xml/xmlutil",
"service/s3",
"service/sts",
"service/sts/stsiface",
]
pruneopts = "UT"
revision = "2eb9a651f22769d792f974e8a8efe16afea3370f"
version = "v1.23.9"
[[projects]]
digest = "1:d6afaeed1502aa28e80a4ed0981d570ad91b2579193404256ce672ed0a609e0d"
name = "github.com/beorn7/perks"
packages = ["quantile"]
pruneopts = "UT"
revision = "4b2b341e8d7715fae06375aa633dbb6e91b3fb46"
version = "v1.0.0"
[[projects]]
digest = "1:fed1f537c2f1269fe475a8556c393fe466641682d73ef8fd0491cd3aa1e47bad"
name = "github.com/certifi/gocertifi"
packages = ["."]
pruneopts = "UT"
revision = "deb3ae2ef2610fde3330947281941c562861188b"
version = "2018.01.18"
[[projects]]
digest = "1:e5003c19d396d8b3cf1324ea0bf49b00f13e9466d0297d1268b641f1c617c3a2"
name = "github.com/cloudflare/brotli-go"
packages = ["."]
pruneopts = "T"
revision = "18c9f6c67e3dfc12e0ddaca748d2887f97a7ac28"
[[projects]]
digest = "1:6dbb2bbc7e6333e691c4d82fd86485f0695a35902fbb9b2df5f72e22ab0040f3"
name = "github.com/cloudflare/golibs"
packages = ["lrucache"]
pruneopts = "UT"
revision = "333127dbecfcc23a8db7d9a4f52785d23aff44a1"
[[projects]]
digest = "1:3f9506ee991cdee1f05bf0cd3e34b5cd922dc00d6a950fb4beb4e07ab1c4d3d1"
name = "github.com/coredns/coredns"
packages = [
"core/dnsserver",
"coremain",
"pb",
"plugin",
"plugin/cache",
"plugin/cache/freq",
"plugin/etcd/msg",
"plugin/metrics",
"plugin/metrics/vars",
"plugin/pkg/cache",
"plugin/pkg/dnstest",
"plugin/pkg/dnsutil",
"plugin/pkg/doh",
"plugin/pkg/edns",
"plugin/pkg/fuzz",
"plugin/pkg/log",
"plugin/pkg/nonwriter",
"plugin/pkg/rcode",
"plugin/pkg/response",
"plugin/pkg/trace",
"plugin/pkg/uniq",
"plugin/pkg/watch",
"plugin/test",
"request",
]
pruneopts = "UT"
revision = "2e322f6e8a54f18c6aef9c25a7c432c291a3d9f7"
version = "v1.2.0"
[[projects]]
digest = "1:6f70106e7bc1c803e8a0a4519e09c12d154771acfa2559206e97b033bbd1dd38"
name = "github.com/coreos/go-oidc"
packages = ["jose"]
pruneopts = "UT"
revision = "a93f71fdfe73d2c0f5413c0565eea0af6523a6df"
[[projects]]
digest = "1:1da3a221f0bc090792d3a2a080ff09008427c0e0f0533a4ed6abd8994421da73"
name = "github.com/coreos/go-systemd"
packages = ["daemon"]
pruneopts = "UT"
revision = "95778dfbb74eb7e4dbaf43bf7d71809650ef8076"
version = "v19"
[[projects]]
digest = "1:385dff5bf83bf1c9d40ef2b0f2e697f0d1973fc9e24ae9401cf41698612bbebc"
name = "github.com/creack/pty"
packages = ["."]
pruneopts = "UT"
revision = "2769f65a3a94eb8f876f44a0459d24ae7ad2e488"
version = "v1.1.7"
[[projects]]
digest = "1:ffe9824d294da03b391f44e1ae8281281b4afc1bdaa9588c9097785e3af10cec"
name = "github.com/davecgh/go-spew"
packages = ["spew"]
pruneopts = "UT"
revision = "8991bc29aa16c548c550c7ff78260e27b9ab7c73"
version = "v1.1.1"
[[projects]]
branch = "master"
digest = "1:c013ffc6e15f9f898078f9d38441c68b228aa7b899659452170250ccb27f5f1e"
name = "github.com/elgs/gosqljson"
packages = ["."]
pruneopts = "UT"
revision = "027aa4915315a0b2825c0f025cea347829b974fa"
[[projects]]
digest = "1:d4268b2a09b1f736633577c4ac93f2a5356c73742fff5344e2451aeec60a7ad0"
name = "github.com/equinox-io/equinox"
packages = [
".",
"internal/go-update",
"internal/go-update/internal/binarydist",
"internal/go-update/internal/osext",
"internal/osext",
"proto",
]
pruneopts = "UT"
revision = "5205c98a6c11dc72747ce12fff6cd620a99fde05"
version = "v1.2.0"
[[projects]]
digest = "1:433763f10d88dba9b533a7ea2fe9f5ee11e57e00306eb97a1f6090fd978e8fa1"
name = "github.com/facebookgo/grace"
packages = ["gracenet"]
pruneopts = "UT"
revision = "75cf19382434e82df4dd84953f566b8ad23d6e9e"
[[projects]]
branch = "master"
digest = "1:50a46ab1d5edbbdd55125b4d37f1bf503d0807c26461f9ad7b358d6006641d09"
name = "github.com/flynn/go-shlex"
packages = ["."]
pruneopts = "UT"
revision = "3f9db97f856818214da2e1057f8ad84803971cff"
[[projects]]
digest = "1:d4623fc7bf7e281d9107367cc4a9e76ed3e86b1eec1a4e30630c870bef1fedd0"
name = "github.com/getsentry/raven-go"
packages = ["."]
pruneopts = "UT"
revision = "ed7bcb39ff10f39ab08e317ce16df282845852fa"
[[projects]]
digest = "1:0a01355005024757ae2a1f62e8fe68a30b4f5fde7a08ac88b3685e7999ac354b"
name = "github.com/gliderlabs/ssh"
packages = ["."]
pruneopts = "UT"
revision = "5b6cc7030f17095c0cf23bb063b0bfe824fe5f8b"
version = "v0.2.2"
[[projects]]
branch = "master"
digest = "1:3e6afc3ed8a72949aa735c00fddc23427dc9384ccfd51cf0d91a412e668da632"
name = "github.com/golang-collections/collections"
packages = ["queue"]
pruneopts = "UT"
revision = "604e922904d35e97f98a774db7881f049cd8d970"
[[projects]]
digest = "1:239c4c7fd2159585454003d9be7207167970194216193a8a210b8d29576f19c9"
name = "github.com/golang/protobuf"
packages = [
"proto",
"ptypes",
"ptypes/any",
"ptypes/duration",
"ptypes/timestamp",
]
pruneopts = "UT"
revision = "b5d812f8a3706043e23a9cd5babf2e5423744d30"
version = "v1.3.1"
[[projects]]
digest = "1:582b704bebaa06b48c29b0cec224a6058a09c86883aaddabde889cd1a5f73e1b"
name = "github.com/google/uuid"
packages = ["."]
pruneopts = "UT"
revision = "0cd6bf5da1e1c83f8b45653022c74f71af0538a4"
version = "v1.1.1"
[[projects]]
digest = "1:d5f97fc268267ec1b61c3453058c738246fc3e746f14b1ae25161513b7367b0c"
name = "github.com/gorilla/mux"
packages = ["."]
pruneopts = "UT"
revision = "c5c6c98bc25355028a63748a498942a6398ccd22"
version = "v1.7.1"
[[projects]]
digest = "1:43dd08a10854b2056e615d1b1d22ac94559d822e1f8b6fcc92c1a1057e85188e"
name = "github.com/gorilla/websocket"
packages = ["."]
pruneopts = "UT"
revision = "ea4d1f681babbce9545c9c5f3d5194a789c89f5b"
version = "v1.2.0"
[[projects]]
branch = "master"
digest = "1:1a1206efd03a54d336dce7bb8719e74f2f8932f661cb9f57d5813a1d99c083d8"
name = "github.com/grpc-ecosystem/grpc-opentracing"
packages = ["go/otgrpc"]
pruneopts = "UT"
revision = "8e809c8a86450a29b90dcc9efbf062d0fe6d9746"
[[projects]]
digest = "1:bb81097a5b62634f3e9fec1014657855610c82d19b9a40c17612e32651e35dca"
name = "github.com/jmespath/go-jmespath"
packages = ["."]
pruneopts = "UT"
revision = "c2b33e84"
[[projects]]
digest = "1:6c41d4f998a03b6604227ccad36edaed6126c397e5d78709ef4814a1145a6757"
name = "github.com/jmoiron/sqlx"
packages = [
".",
"reflectx",
]
pruneopts = "UT"
revision = "d161d7a76b5661016ad0b085869f77fd410f3e6a"
version = "v1.2.0"
[[projects]]
digest = "1:31e761d97c76151dde79e9d28964a812c46efc5baee4085b86f68f0c654450de"
name = "github.com/konsorten/go-windows-terminal-sequences"
packages = ["."]
pruneopts = "UT"
revision = "f55edac94c9bbba5d6182a4be46d86a2c9b5b50e"
version = "v1.0.2"
[[projects]]
digest = "1:bc1c0be40c67b6b4aee09d7508d5a2a52c1c116b1fa43806dad2b0d6b4d4003b"
name = "github.com/lib/pq"
packages = [
".",
"oid",
"scram",
]
pruneopts = "UT"
revision = "51e2106eed1cea199c802d2a49e91e2491b02056"
version = "v1.1.0"
[[projects]]
digest = "1:2fa7b0155cd54479a755c629de26f888a918e13f8857a2c442205d825368e084"
name = "github.com/mattn/go-colorable"
packages = ["."]
pruneopts = "UT"
revision = "3a70a971f94a22f2fa562ffcc7a0eb45f5daf045"
version = "v0.1.1"
[[projects]]
digest = "1:e150b5fafbd7607e2d638e4e5cf43aa4100124e5593385147b0a74e2733d8b0d"
name = "github.com/mattn/go-isatty"
packages = ["."]
pruneopts = "UT"
revision = "c2a7a6ca930a4cd0bc33a3f298eb71960732a3a7"
version = "v0.0.7"
[[projects]]
digest = "1:ff5ebae34cfbf047d505ee150de27e60570e8c394b3b8fdbb720ff6ac71985fc"
name = "github.com/matttproud/golang_protobuf_extensions"
packages = ["pbutil"]
pruneopts = "UT"
revision = "c12348ce28de40eed0136aa2b644d0ee0650e56c"
version = "v1.0.1"
[[projects]]
digest = "1:75fa16a231ef40da3e462d651c20b9df20bde0777bdc1ac0982242c79057ee71"
name = "github.com/mholt/caddy"
packages = [
".",
"caddyfile",
"telemetry",
]
pruneopts = "UT"
revision = "d3b731e9255b72d4571a5aac125634cf1b6031dc"
[[projects]]
digest = "1:2b4b4b2e5544c2a11a486c1b631357aa2ddf766e50c1b2483cf809da2c511234"
name = "github.com/miekg/dns"
packages = ["."]
pruneopts = "UT"
revision = "73601d4aed9d844322611759d7f3619110b7c88e"
version = "v1.1.8"
[[projects]]
digest = "1:5d231480e1c64a726869bc4142d270184c419749d34f167646baa21008eb0a79"
name = "github.com/mitchellh/go-homedir"
packages = ["."]
pruneopts = "UT"
revision = "af06845cf3004701891bf4fdb884bfe4920b3727"
version = "v1.1.0"
[[projects]]
digest = "1:53bc4cd4914cd7cd52139990d5170d6dc99067ae31c56530621b18b35fc30318"
name = "github.com/mitchellh/mapstructure"
packages = ["."]
pruneopts = "UT"
revision = "3536a929edddb9a5b34bd6861dc4a9647cb459fe"
version = "v1.1.2"
[[projects]]
digest = "1:11e62d6050198055e6cd87ed57e5d8c669e84f839c16e16f192374d913d1a70d"
name = "github.com/opentracing/opentracing-go"
packages = [
".",
"ext",
"log",
]
pruneopts = "UT"
revision = "659c90643e714681897ec2521c60567dd21da733"
version = "v1.1.0"
[[projects]]
digest = "1:40e195917a951a8bf867cd05de2a46aaf1806c50cf92eebf4c16f78cd196f747"
name = "github.com/pkg/errors"
packages = ["."]
pruneopts = "UT"
revision = "645ef00459ed84a119197bfb8d8205042c6df63d"
version = "v0.8.0"
[[projects]]
digest = "1:0028cb19b2e4c3112225cd871870f2d9cf49b9b4276531f03438a88e94be86fe"
name = "github.com/pmezard/go-difflib"
packages = ["difflib"]
pruneopts = "UT"
revision = "792786c7400a136282c1664665ae0a8db921c6c2"
version = "v1.0.0"
[[projects]]
digest = "1:c968b29db5d68ec97de404b6d058d5937fa015a141b3b4f7a0d87d5f8226f04c"
name = "github.com/prometheus/client_golang"
packages = [
"prometheus",
"prometheus/promhttp",
]
pruneopts = "UT"
revision = "967789050ba94deca04a5e84cce8ad472ce313c1"
version = "v0.9.0-pre1"
[[projects]]
branch = "master"
digest = "1:2d5cd61daa5565187e1d96bae64dbbc6080dacf741448e9629c64fd93203b0d4"
name = "github.com/prometheus/client_model"
packages = ["go"]
pruneopts = "UT"
revision = "fd36f4220a901265f90734c3183c5f0c91daa0b8"
[[projects]]
digest = "1:35cf6bdf68db765988baa9c4f10cc5d7dda1126a54bd62e252dbcd0b1fc8da90"
name = "github.com/prometheus/common"
packages = [
"expfmt",
"internal/bitbucket.org/ww/goautoneg",
"model",
]
pruneopts = "UT"
revision = "a82f4c12f983cc2649298185f296632953e50d3e"
version = "v0.3.0"
[[projects]]
branch = "master"
digest = "1:49b09905e781d7775c086604cc00083e1832d0783f1f421b79f42657c457d029"
name = "github.com/prometheus/procfs"
packages = ["."]
pruneopts = "UT"
revision = "8368d24ba045f26503eb745b624d930cbe214c79"
[[projects]]
digest = "1:1a23fdd843129ef761ffe7651bc5fe7c5b09fbe933e92783ab06cc11c37b7b37"
name = "github.com/rifflock/lfshook"
packages = ["."]
pruneopts = "UT"
revision = "b9218ef580f59a2e72dad1aa33d660150445d05a"
version = "v2.4"
[[projects]]
digest = "1:31538f8d774e8bf2fb9380d2d2a82d69e206a7d0603946586926f1819c546c26"
name = "github.com/sirupsen/logrus"
packages = [
".",
"hooks/test",
]
pruneopts = "UT"
revision = "839c75faf7f98a33d445d181f3018b5c3409a45e"
version = "v1.4.2"
[[projects]]
digest = "1:7e8d267900c7fa7f35129a2a37596e38ed0f11ca746d6d9ba727980ee138f9f6"
name = "github.com/stretchr/testify"
packages = [
"assert",
"require",
]
pruneopts = "UT"
revision = "12b6f73e6084dad08a7c6e575284b177ecafbc71"
version = "v1.2.1"
[[projects]]
branch = "master"
digest = "1:a84d5ec8b40a827962ea250f2cf03434138ccae9d83fcac12fb49b70c70b80cc"
name = "golang.org/x/crypto"
packages = [
"curve25519",
"ed25519",
"ed25519/internal/edwards25519",
"internal/chacha20",
"internal/subtle",
"nacl/box",
"nacl/secretbox",
"poly1305",
"salsa20/salsa",
"ssh",
"ssh/terminal",
]
pruneopts = "UT"
revision = "f416ebab96af27ca70b6e5c23d6a0747530da626"
[[projects]]
branch = "master"
digest = "1:52d140f7ab52e491cc1cbc93e6637aa5e9a7f3beae7545d675b02e52ca9d7290"
name = "golang.org/x/net"
packages = [
"bpf",
"context",
"http/httpguts",
"http2",
"http2/hpack",
"idna",
"internal/iana",
"internal/socket",
"internal/timeseries",
"ipv4",
"ipv6",
"trace",
"websocket",
]
pruneopts = "UT"
revision = "1da14a5a36f220ea3f03470682b737b1dfd5de22"
[[projects]]
digest = "1:39ebcc2b11457b703ae9ee2e8cca0f68df21969c6102cb3b705f76cca0ea0239"
name = "golang.org/x/sync"
packages = ["errgroup"]
pruneopts = "UT"
revision = "1d60e4601c6fd243af51cc01ddf169918a5407ca"
[[projects]]
branch = "master"
digest = "1:77751d02e939d7078faedaeec10c09af575a09c528d84d18f2cb45a84bd1889a"
name = "golang.org/x/sys"
packages = [
"cpu",
"unix",
"windows",
"windows/registry",
"windows/svc",
"windows/svc/eventlog",
"windows/svc/mgr",
]
pruneopts = "UT"
revision = "12500544f89f9420afe9529ba8940bf72d294972"
[[projects]]
digest = "1:a2ab62866c75542dd18d2b069fec854577a20211d7c0ea6ae746072a1dccdd18"
name = "golang.org/x/text"
packages = [
"collate",
"collate/build",
"internal/colltab",
"internal/gen",
"internal/tag",
"internal/triegen",
"internal/ucd",
"language",
"secure/bidirule",
"transform",
"unicode/bidi",
"unicode/cldr",
"unicode/norm",
"unicode/rangetable",
]
pruneopts = "UT"
revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
version = "v0.3.0"
[[projects]]
branch = "master"
digest = "1:c3076e7defee87de1236f1814beb588f40a75544c60121e6eb38b3b3721783e2"
name = "google.golang.org/genproto"
packages = ["googleapis/rpc/status"]
pruneopts = "UT"
revision = "d1146b9035b912113a38af3b138eb2af567b2c67"
[[projects]]
digest = "1:31d87f39886fb38a2b6c097ff3b9f985d6960772170d64a68246f7790e955746"
name = "google.golang.org/grpc"
packages = [
".",
"balancer",
"balancer/base",
"balancer/roundrobin",
"binarylog/grpc_binarylog_v1",
"codes",
"connectivity",
"credentials",
"credentials/internal",
"encoding",
"encoding/proto",
"grpclog",
"internal",
"internal/backoff",
"internal/balancerload",
"internal/binarylog",
"internal/channelz",
"internal/envconfig",
"internal/grpcrand",
"internal/grpcsync",
"internal/syscall",
"internal/transport",
"keepalive",
"metadata",
"naming",
"peer",
"resolver",
"resolver/dns",
"resolver/passthrough",
"stats",
"status",
"tap",
]
pruneopts = "UT"
revision = "236199dd5f8031d698fb64091194aecd1c3895b2"
version = "v1.20.0"
[[projects]]
branch = "altsrc-parse-durations"
digest = "1:0370b1bceda03dbfade3abbde639a43f1113bab711ec760452e5c0dcc0c14787"
name = "gopkg.in/urfave/cli.v2"
packages = [
".",
"altsrc",
]
pruneopts = "UT"
revision = "d604b6ffeee878fbf084fd2761466b6649989cee"
source = "https://github.com/cbranch/cli"
[[projects]]
digest = "1:4d2e5a73dc1500038e504a8d78b986630e3626dc027bc030ba5c75da257cdb96"
name = "gopkg.in/yaml.v2"
packages = ["."]
pruneopts = "UT"
revision = "51d6538a90f86fe93ac480b35f37b2be17fef232"
version = "v2.2.2"
[[projects]]
digest = "1:8ffc3ddc31414c0a71220957bb723b16510d7fcb5b3880dc0da4cf6d39c31642"
name = "zombiezen.com/go/capnproto2"
packages = [
".",
"encoding/text",
"internal/fulfiller",
"internal/nodemap",
"internal/packed",
"internal/queue",
"internal/schema",
"internal/strquote",
"pogs",
"rpc",
"rpc/internal/refcount",
"schemas",
"server",
"std/capnp/rpc",
]
pruneopts = "UT"
revision = "7cfd211c19c7f5783c695f3654efa46f0df259c3"
source = "https://github.com/zombiezen/go-capnproto2"
version = "v2.17.1"
[solve-meta]
analyzer-name = "dep"
analyzer-version = 1
input-imports = [
"github.com/DATA-DOG/go-sqlmock",
"github.com/aws/aws-sdk-go/aws",
"github.com/aws/aws-sdk-go/aws/credentials",
"github.com/aws/aws-sdk-go/aws/session",
"github.com/aws/aws-sdk-go/service/s3",
"github.com/cloudflare/brotli-go",
"github.com/cloudflare/golibs/lrucache",
"github.com/coredns/coredns/core/dnsserver",
"github.com/coredns/coredns/plugin",
"github.com/coredns/coredns/plugin/cache",
"github.com/coredns/coredns/plugin/metrics/vars",
"github.com/coredns/coredns/plugin/pkg/dnstest",
"github.com/coredns/coredns/plugin/pkg/rcode",
"github.com/coredns/coredns/request",
"github.com/coreos/go-oidc/jose",
"github.com/coreos/go-systemd/daemon",
"github.com/creack/pty",
"github.com/elgs/gosqljson",
"github.com/equinox-io/equinox",
"github.com/facebookgo/grace/gracenet",
"github.com/getsentry/raven-go",
"github.com/gliderlabs/ssh",
"github.com/golang-collections/collections/queue",
"github.com/google/uuid",
"github.com/gorilla/mux",
"github.com/gorilla/websocket",
"github.com/lib/pq",
"github.com/mattn/go-colorable",
"github.com/miekg/dns",
"github.com/mitchellh/go-homedir",
"github.com/mitchellh/mapstructure",
"github.com/pkg/errors",
"github.com/prometheus/client_golang/prometheus",
"github.com/prometheus/client_golang/prometheus/promhttp",
"github.com/rifflock/lfshook",
"github.com/sirupsen/logrus",
"github.com/sirupsen/logrus/hooks/test",
"github.com/stretchr/testify/assert",
"github.com/stretchr/testify/require",
"golang.org/x/crypto/nacl/box",
"golang.org/x/crypto/ssh",
"golang.org/x/crypto/ssh/terminal",
"golang.org/x/net/context",
"golang.org/x/net/http2",
"golang.org/x/net/http2/hpack",
"golang.org/x/net/idna",
"golang.org/x/net/trace",
"golang.org/x/net/websocket",
"golang.org/x/sync/errgroup",
"golang.org/x/sys/windows",
"golang.org/x/sys/windows/svc",
"golang.org/x/sys/windows/svc/eventlog",
"golang.org/x/sys/windows/svc/mgr",
"gopkg.in/urfave/cli.v2",
"gopkg.in/urfave/cli.v2/altsrc",
"zombiezen.com/go/capnproto2",
"zombiezen.com/go/capnproto2/encoding/text",
"zombiezen.com/go/capnproto2/pogs",
"zombiezen.com/go/capnproto2/rpc",
"zombiezen.com/go/capnproto2/schemas",
"zombiezen.com/go/capnproto2/server",
"zombiezen.com/go/capnproto2/std/capnp/rpc",
]
solver-name = "gps-cdcl"
solver-version = 1
-103
View File
@@ -1,103 +0,0 @@
[prune]
go-tests = true
unused-packages = true
[[prune.project]]
name = "github.com/cloudflare/brotli-go"
unused-packages = false
[[constraint]]
name = "github.com/facebookgo/grace"
revision = "75cf19382434e82df4dd84953f566b8ad23d6e9e"
[[constraint]]
name = "github.com/getsentry/raven-go"
revision = "ed7bcb39ff10f39ab08e317ce16df282845852fa"
[[constraint]]
name = "github.com/pkg/errors"
version = "=0.8.0"
[[constraint]]
name = "github.com/prometheus/client_golang"
version = "=0.9.0-pre1"
[[constraint]]
name = "github.com/sirupsen/logrus"
version = "=1.4.2"
[[constraint]]
name = "github.com/stretchr/testify"
version = "=1.2.1"
[[constraint]]
name = "golang.org/x/net"
branch = "master" # master required by github.com/miekg/dns
[[constraint]]
name = "golang.org/x/sync"
revision = "1d60e4601c6fd243af51cc01ddf169918a5407ca"
[[constraint]]
name = "gopkg.in/urfave/cli.v2"
source = "https://github.com/cbranch/cli"
branch = "altsrc-parse-durations"
[[constraint]]
name = "zombiezen.com/go/capnproto2"
source = "https://github.com/zombiezen/go-capnproto2"
version = "=2.17.1"
[[constraint]]
name = "github.com/gorilla/websocket"
version = "=1.2.0"
[[constraint]]
name = "github.com/coredns/coredns"
version = "=1.2.0"
[[constraint]]
name = "github.com/miekg/dns"
version = "=1.1.8"
[[constraint]]
name = "github.com/cloudflare/brotli-go"
revision = "18c9f6c67e3dfc12e0ddaca748d2887f97a7ac28"
[[override]]
name = "github.com/mholt/caddy"
revision = "d3b731e9255b72d4571a5aac125634cf1b6031dc"
[[constraint]]
name = "github.com/coreos/go-oidc"
revision = "a93f71fdfe73d2c0f5413c0565eea0af6523a6df"
[[constraint]]
name = "golang.org/x/crypto"
branch = "master" # master required by github.com/miekg/dns
[[constraint]]
name = "github.com/cloudflare/golibs"
revision = "333127dbecfcc23a8db7d9a4f52785d23aff44a1"
[[constraint]]
name = "github.com/google/uuid"
version = "=1.1.1"
[[constraint]]
name = "github.com/mitchellh/mapstructure"
version = "1.1.2"
[[constraint]]
name = "github.com/gliderlabs/ssh"
version = "0.2.2"
[[constraint]]
name = "github.com/creack/pty"
version = "1.1.7"
[[constraint]]
name = "github.com/aws/aws-sdk-go"
version = "1.23.9"
+8 -4
View File
@@ -29,7 +29,7 @@ clean:
.PHONY: cloudflared
cloudflared: tunnel-deps
go build -v $(VERSION_FLAGS) $(IMPORT_PATH)/cmd/cloudflared
go build -v -mod=vendor $(VERSION_FLAGS) $(IMPORT_PATH)/cmd/cloudflared
.PHONY: container
container:
@@ -37,7 +37,11 @@ container:
.PHONY: test
test: vet
go test -v -race $(VERSION_FLAGS) ./...
go test -v -mod=vendor -race $(VERSION_FLAGS) ./...
.PHONY: test-ssh-server
test-ssh-server:
docker-compose -f ssh_server_tests/docker-compose.yml up
.PHONY: cloudflared-deb
cloudflared-deb: cloudflared
@@ -78,6 +82,6 @@ tunnelrpc/tunnelrpc.capnp.go: tunnelrpc/tunnelrpc.capnp
.PHONY: vet
vet:
go vet ./...
go vet -mod=vendor ./...
which go-sumtype # go get github.com/BurntSushi/go-sumtype
go-sumtype $$(go list ./...)
go-sumtype $$(go list -mod=vendor ./...)
+67
View File
@@ -1,3 +1,70 @@
2019.11.2
- 2019-11-18 TUN-2567: AuthOutcome can be turned back into AuthResponse
- 2019-11-18 TUN-2563: Exposes config_version metrics
2019.11.1
- 2019-11-12 Add db-connect, a SQL over HTTPS server
- 2019-11-12 TUN-2053: Add a /healthcheck endpoint to the metrics server
- 2019-11-13 TUN-2178: public API to create new h2mux.MuxedStreamRequest
- 2019-11-13 TUN-2490: respect original representation of HTTP request path
- 2019-11-18 TUN-2547: TunnelRPC definitions for Authenticate flow
- 2019-11-18 TUN-2551: TunnelRPC definitions for ReconnectTunnel flow
- 2019-11-05 TUN-2506: Expose active streams metrics
2019.11.0
- 2019-11-04 TUN-2502: Switch to go modules
- 2019-11-04 TUN-2500: Don't send client registration errors to Sentry
- 2019-11-04 TUN-2489: Delete stream from activestreammap when read and write are both closed
- 2019-11-05 TUN-2505: Terminate stream on receipt of RST_STREAM; MuxedStream.CloseWrite() should terminate the MuxedStream.Write() loop
- 2019-10-30 TUN-2451: Log inavlid path
- 2019-10-22 TUN-2425: Enable cloudflared to serve multiple Hello World servers by having each of them create its own ServeMux
- 2019-10-22 AUTH-2173: Prepends access login url with scheme if one doesnt exist
- 2019-10-23 TUN-2460: Configure according to the ClientConfig recevied from a successful Connect
- 2019-10-23 AUTH-2177: Reads and writes error streams
2019.10.4
- 2019-10-21 TUN-2450: Remove Brew publishing formula
2019.10.3
- 2019-10-18 Fix #129: Excessive memory usage streaming large files (#142)
2019.10.2
- 2019-10-17 AUTH-2167: Adds CLI option for host key directory
2019.10.1
- 2019-10-17 Adds variable to fix windows build
2019.10.0
- 2019-10-11 AUTH-2105: Dont require --destination arg
- 2019-10-14 TUN-2344: log more details: http2.Framer.ErrorDetail() if available, connectionID
- 2019-10-16 AUTH-2159: Moves shutdownC close into error handling AUTH-2161: Lowers size of preamble length AUTH-2160: Fixes url parsing logic
- 2019-10-16 AUTH-2135: Adds support for IPv6 and tests
- 2019-10-02 AUTH-2105: Adds support for local forwarding. Refactor auditlogger creation. AUTH-2088: Adds dynamic destination routing
- 2019-10-09 AUTH-2114: Uses short lived cert auth for outgoing client connection
- 2019-09-30 AUTH-2089: Revise ssh server to function as a proxy
2019.9.2
- 2019-09-26 TUN-2355: Roll back TUN-2276
2019.9.1
- 2019-09-23 TUN-2334: remove tlsConfig.ServerName special case
- 2019-09-23 AUTH-2077: Quotes open browser command in windows
- 2019-09-11 AUTH-2050: Adds time.sleep to temporarily avoid hitting tunnel muxer dealock issue
- 2019-09-10 AUTH-2056: Writes stderr to its own stream for non-pty connections
- 2019-09-16 TUN-2307: Capnp is the only serialization format used in tunnelpogs
- 2019-09-18 TUN-2315: Replace Scope with IntentLabel
- 2019-09-17 TUN-2309: Split ConnectResult into ConnectError and ConnectSuccess, each implementing its own capnp serialization logic
- 2019-09-18 AUTH-2052: Adds tests for SSH server
- 2019-09-18 AUTH-2067: Log commands correctly
- 2019-09-19 AUTH-2055: Verifies token at edge on access login
- 2019-09-04 TUN-2201: change SRV records used by cloudflared
- 2019-09-06 TUN-2280: Revert "TUN-2260: add name/group to CapnpConnectParameters, remove Scope"
- 2019-09-03 AUTH-1943 hooked up uploader to logger, added timestamp to session logs, add tests
- 2019-09-04 AUTH-2036: Refactor user retrieval, shutdown after ssh server stops, add custom version string
- 2019-09-06 AUTH-1942 added event log to ssh server
- 2019-09-04 AUTH-2037: Adds support for ssh port forwarding
- 2019-09-05 TUN-2276: Path encoding broken
2019.9.0
- 2019-09-05 TUN-2279: Revert path encoding fix
- 2019-08-30 AUTH-2021 - check error for failing tests
+1 -1
View File
@@ -77,7 +77,7 @@ func (m *DirectoryUploadManager) sweep() {
checkTime := time.Now().Add(-time.Duration(retentionTime))
//delete the file it is stale
if info.ModTime().After(checkTime) {
if info.ModTime().Before(checkTime) {
os.Remove(path)
return nil
}
+7 -7
View File
@@ -114,7 +114,7 @@ func createWebsocketStream(options *StartOptions) (*cloudflaredWebsocket.Conn, e
wsConn, resp, err := cloudflaredWebsocket.ClientConnect(req, nil)
defer closeRespBody(resp)
if err != nil && isAccessResponse(resp) {
if err != nil && IsAccessResponse(resp) {
wsConn, err = createAccessAuthenticatedStream(options)
if err != nil {
return nil, err
@@ -126,10 +126,10 @@ func createWebsocketStream(options *StartOptions) (*cloudflaredWebsocket.Conn, e
return &cloudflaredWebsocket.Conn{Conn: wsConn}, nil
}
// isAccessResponse checks the http Response to see if the url location
// IsAccessResponse checks the http Response to see if the url location
// contains the Access structure.
func isAccessResponse(resp *http.Response) bool {
if resp == nil || resp.StatusCode <= 300 {
func IsAccessResponse(resp *http.Response) bool {
if resp == nil || resp.StatusCode != http.StatusFound {
return false
}
@@ -156,7 +156,7 @@ func createAccessAuthenticatedStream(options *StartOptions) (*websocket.Conn, er
return wsConn, nil
}
if !isAccessResponse(resp) {
if !IsAccessResponse(resp) {
return nil, err
}
@@ -179,7 +179,7 @@ func createAccessAuthenticatedStream(options *StartOptions) (*websocket.Conn, er
// createAccessWebSocketStream builds an Access request and makes a connection
func createAccessWebSocketStream(options *StartOptions) (*websocket.Conn, *http.Response, error) {
req, err := buildAccessRequest(options)
req, err := BuildAccessRequest(options)
if err != nil {
return nil, nil, err
}
@@ -187,7 +187,7 @@ func createAccessWebSocketStream(options *StartOptions) (*websocket.Conn, *http.
}
// buildAccessRequest builds an HTTP request with the Access token set
func buildAccessRequest(options *StartOptions) (*http.Request, error) {
func BuildAccessRequest(options *StartOptions) (*http.Request, error) {
req, err := http.NewRequest(http.MethodGet, options.OriginURL, nil)
if err != nil {
return nil, err
+4 -4
View File
@@ -102,14 +102,14 @@ func TestIsAccessResponse(t *testing.T) {
ExpectedOut bool
}{
{"nil response", nil, false},
{"redirect with no location", &http.Response{StatusCode: http.StatusPermanentRedirect}, false},
{"redirect with no location", &http.Response{StatusCode: http.StatusFound}, false},
{"200 ok", &http.Response{StatusCode: http.StatusOK}, false},
{"redirect with location", &http.Response{StatusCode: http.StatusPermanentRedirect, Header: validLocationHeader}, true},
{"redirect with invalid location", &http.Response{StatusCode: http.StatusPermanentRedirect, Header: invalidLocationHeader}, false},
{"redirect with location", &http.Response{StatusCode: http.StatusFound, Header: validLocationHeader}, true},
{"redirect with invalid location", &http.Response{StatusCode: http.StatusFound, Header: invalidLocationHeader}, false},
}
for i, tc := range testCases {
if isAccessResponse(tc.In) != tc.ExpectedOut {
if IsAccessResponse(tc.In) != tc.ExpectedOut {
t.Fatalf("Failed case %d -- %s", i, tc.Description)
}
}
+15 -12
View File
@@ -1,5 +1,5 @@
pinned_go: &pinned_go go=1.12.7-1
build_dir: &build_dir /cfsetup_build/src/github.com/cloudflare/cloudflared/
build_dir: &build_dir /cfsetup_build
default-flavor: stretch
stretch: &stretch
build:
@@ -8,7 +8,6 @@ stretch: &stretch
- *pinned_go
- build-essential
post-cache:
- export GOPATH=/cfsetup_build/
- export GOOS=linux
- export GOARCH=amd64
- make cloudflared
@@ -20,7 +19,6 @@ stretch: &stretch
- fakeroot
- rubygem-fpm
post-cache:
- export GOPATH=/cfsetup_build/
- export GOOS=linux
- export GOARCH=amd64
- make cloudflared-deb
@@ -30,7 +28,6 @@ stretch: &stretch
- *pinned_go
- build-essential
post-cache:
- export GOPATH=/cfsetup_build/
- export GOOS=linux
- export GOARCH=amd64
- make release
@@ -41,7 +38,6 @@ stretch: &stretch
- crossbuild-essential-armhf
- gcc-arm-linux-gnueabihf
post-cache:
- export GOPATH=/cfsetup_build/
- export GOOS=linux
- export GOARCH=arm
- export CC=arm-linux-gnueabihf-gcc
@@ -52,7 +48,6 @@ stretch: &stretch
- *pinned_go
- gcc-multilib
post-cache:
- export GOPATH=/cfsetup_build/
- export GOOS=linux
- export GOARCH=386
- make release
@@ -62,7 +57,6 @@ stretch: &stretch
- *pinned_go
- gcc-mingw-w64
post-cache:
- export GOPATH=/cfsetup_build/
- export GOOS=windows
- export GOARCH=amd64
- export CC=x86_64-w64-mingw32-gcc
@@ -73,7 +67,6 @@ stretch: &stretch
- *pinned_go
- gcc-mingw-w64
post-cache:
- export GOPATH=/cfsetup_build/
- export GOOS=windows
- export GOARCH=386
- export CC=i686-w64-mingw32-gcc-win32
@@ -84,12 +77,22 @@ stretch: &stretch
- *pinned_go
- build-essential
post-cache:
- export GOPATH=/cfsetup_build/
- export GOOS=linux
- export GOARCH=amd64
- sudo chown -R $(whoami) /cfsetup_build/
- go get github.com/BurntSushi/go-sumtype
- export PATH="$GOPATH/bin:$PATH"
# cd to a non-module directory: https://github.com/golang/go/issues/24250
- (cd / && go get github.com/BurntSushi/go-sumtype)
- export PATH="$HOME/go/bin:$PATH"
- make test
jessie: *stretch
# cfsetup compose
default-stack: test_dbconnect
test_dbconnect:
compose:
up-args:
- --renew-anon-volumes
- --abort-on-container-exit
- --exit-code-from=cloudflared
files:
- dbconnect_tests/dbconnect.yaml
+6 -1
View File
@@ -23,7 +23,7 @@ func ssh(c *cli.Context) error {
if err != nil || rawHostName == "" {
return cli.ShowCommandHelp(c, "ssh")
}
originURL := "https://" + hostname
originURL := ensureURLScheme(hostname)
// get the headers from the cmdline and add them
headers := buildRequestHeaders(c.StringSlice(sshHeaderFlag))
@@ -34,6 +34,11 @@ func ssh(c *cli.Context) error {
headers.Add("CF-Access-Client-Secret", c.String(sshTokenSecretFlag))
}
destination := c.String(sshDestinationFlag)
if destination != "" {
headers.Add("CF-Access-SSH-Destination", destination)
}
options := &carrier.StartOptions{
OriginURL: originURL,
Headers: headers,
+86 -6
View File
@@ -1,17 +1,20 @@
package access
import (
"errors"
"fmt"
"net/http"
"net/url"
"os"
"strings"
"text/template"
"time"
"github.com/cloudflare/cloudflared/carrier"
"github.com/cloudflare/cloudflared/cmd/cloudflared/shell"
"github.com/cloudflare/cloudflared/cmd/cloudflared/token"
"github.com/cloudflare/cloudflared/sshgen"
"github.com/cloudflare/cloudflared/validation"
"github.com/pkg/errors"
"golang.org/x/net/idna"
"github.com/cloudflare/cloudflared/log"
@@ -21,6 +24,7 @@ import (
const (
sshHostnameFlag = "hostname"
sshDestinationFlag = "destination"
sshURLFlag = "url"
sshHeaderFlag = "header"
sshTokenIDFlag = "service-token-id"
@@ -124,6 +128,10 @@ func Commands() []*cli.Command {
Name: sshHostnameFlag,
Usage: "specify the hostname of your application.",
},
&cli.StringFlag{
Name: sshDestinationFlag,
Usage: "specify the destination address of your SSH server.",
},
&cli.StringFlag{
Name: sshURLFlag,
Usage: "specify the host:port to forward data to Cloudflare edge.",
@@ -183,14 +191,20 @@ func login(c *cli.Context) error {
raven.SetDSN(sentryDSN)
logger := log.CreateLogger()
args := c.Args()
appURL, err := url.Parse(args.First())
rawURL := ensureURLScheme(args.First())
appURL, err := url.Parse(rawURL)
if args.Len() < 1 || err != nil {
logger.Errorf("Please provide the url of the Access application\n")
return err
}
token, err := token.FetchToken(appURL)
if err != nil {
logger.Errorf("Failed to fetch token: %s\n", err)
if err := verifyTokenAtEdge(appURL, c); err != nil {
logger.WithError(err).Error("Could not verify token")
return err
}
token, err := token.GetTokenIfExists(appURL)
if err != nil || token == "" {
fmt.Fprintln(os.Stderr, "Unable to find token for provided application.")
return err
}
fmt.Fprintf(os.Stdout, "Successfully fetched your token:\n\n%s\n\n", string(token))
@@ -198,6 +212,16 @@ func login(c *cli.Context) error {
return nil
}
// ensureURLScheme prepends a URL with https:// if it doesnt have a scheme. http:// URLs will not be converted.
func ensureURLScheme(url string) string {
url = strings.Replace(strings.ToLower(url), "http://", "https://", 1)
if !strings.HasPrefix(url, "https://") {
url = fmt.Sprintf("https://%s", url)
}
return url
}
// curl provides a wrapper around curl, passing Access JWT along in request
func curl(c *cli.Context) error {
raven.SetDSN(sentryDSN)
@@ -281,7 +305,7 @@ func sshGen(c *cli.Context) error {
return cli.ShowCommandHelp(c, "ssh-gen")
}
originURL, err := url.Parse("https://" + hostname)
originURL, err := url.Parse(ensureURLScheme(hostname))
if err != nil {
return err
}
@@ -372,3 +396,59 @@ func isFileThere(candidate string) bool {
}
return true
}
// verifyTokenAtEdge checks for a token on disk, or generates a new one.
// Then makes a request to to the origin with the token to ensure it is valid.
// Returns nil if token is valid.
func verifyTokenAtEdge(appUrl *url.URL, c *cli.Context) error {
headers := buildRequestHeaders(c.StringSlice(sshHeaderFlag))
if c.IsSet(sshTokenIDFlag) {
headers.Add("CF-Access-Client-Id", c.String(sshTokenIDFlag))
}
if c.IsSet(sshTokenSecretFlag) {
headers.Add("CF-Access-Client-Secret", c.String(sshTokenSecretFlag))
}
options := &carrier.StartOptions{OriginURL: appUrl.String(), Headers: headers}
if valid, err := isTokenValid(options); err != nil {
return err
} else if valid {
return nil
}
if err := token.RemoveTokenIfExists(appUrl); err != nil {
return err
}
if valid, err := isTokenValid(options); err != nil {
return err
} else if !valid {
return errors.New("failed to verify token")
}
return nil
}
// isTokenValid makes a request to the origin and returns true if the response was not a 302.
func isTokenValid(options *carrier.StartOptions) (bool, error) {
req, err := carrier.BuildAccessRequest(options)
if err != nil {
return false, errors.Wrap(err, "Could not create access request")
}
// Do not follow redirects
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
Timeout: time.Second * 5,
}
resp, err := client.Do(req)
if err != nil {
return false, err
}
defer resp.Body.Close()
// A redirect to login means the token was invalid.
return !carrier.IsAccessResponse(resp), nil
}
+25
View File
@@ -0,0 +1,25 @@
package access
import "testing"
func Test_ensureURLScheme(t *testing.T) {
type args struct {
url string
}
tests := []struct {
name string
args args
want string
}{
{"no scheme", args{"localhost:123"}, "https://localhost:123"},
{"http scheme", args{"http://test"}, "https://test"},
{"https scheme", args{"https://test"}, "https://test"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := ensureURLScheme(tt.args.url); got != tt.want {
t.Errorf("ensureURLScheme() = %v, want %v", got, tt.want)
}
})
}
}
@@ -0,0 +1,11 @@
//+build darwin
package shell
import (
"os/exec"
)
func getBrowserCmd(url string) *exec.Cmd {
return exec.Command("open", url)
}
@@ -0,0 +1,11 @@
//+build !windows,!darwin,!linux,!netbsd,!freebsd,!openbsd
package shell
import (
"os/exec"
)
func getBrowserCmd(url string) *exec.Cmd {
return nil
}
@@ -0,0 +1,11 @@
//+build linux freebsd openbsd netbsd
package shell
import (
"os/exec"
)
func getBrowserCmd(url string) *exec.Cmd {
return exec.Command("xdg-open", url)
}
@@ -0,0 +1,18 @@
//+build windows
package shell
import (
"fmt"
"os/exec"
"syscall"
)
func getBrowserCmd(url string) *exec.Cmd {
cmd := exec.Command("cmd")
// CmdLine is only defined when compiling for windows.
// Empty string is the cmd proc "Title". Needs to be included because the start command will interpret the first
// quoted string as that field and we want to quote the URL.
cmd.SysProcAttr = &syscall.SysProcAttr{CmdLine: fmt.Sprintf(`/c start "" "%s"`, url)}
return cmd
}
+1 -15
View File
@@ -4,25 +4,11 @@ import (
"io"
"os"
"os/exec"
"runtime"
)
// OpenBrowser opens the specified URL in the default browser of the user
func OpenBrowser(url string) error {
var cmd string
var args []string
switch runtime.GOOS {
case "windows":
cmd = "cmd"
args = []string{"/c", "start"}
case "darwin":
cmd = "open"
default: // "linux", "freebsd", "openbsd", "netbsd"
cmd = "xdg-open"
}
args = append(args, url)
return exec.Command(cmd, args...).Start()
return getBrowserCmd(url).Start()
}
// Run will kick off a shell task and pipe the results to the respective std pipes
+8 -8
View File
@@ -26,15 +26,15 @@ var logger = log.CreateLogger()
type lock struct {
lockFilePath string
backoff *origin.BackoffHandler
sigHandler *signalHandler
sigHandler *signalHandler
}
type signalHandler struct {
sigChannel chan os.Signal
signals []os.Signal
sigChannel chan os.Signal
signals []os.Signal
}
func (s *signalHandler) register(handler func()){
func (s *signalHandler) register(handler func()) {
s.sigChannel = make(chan os.Signal, 1)
signal.Notify(s.sigChannel, s.signals...)
go func(s *signalHandler) {
@@ -59,8 +59,8 @@ func newLock(path string) *lock {
return &lock{
lockFilePath: lockPath,
backoff: &origin.BackoffHandler{MaxRetries: 7},
sigHandler: &signalHandler{
signals: []os.Signal{syscall.SIGINT, syscall.SIGTERM},
sigHandler: &signalHandler{
signals: []os.Signal{syscall.SIGINT, syscall.SIGTERM},
},
}
}
@@ -68,8 +68,8 @@ func newLock(path string) *lock {
func (l *lock) Acquire() error {
// Intercept SIGINT and SIGTERM to release lock before exiting
l.sigHandler.register(func() {
l.deleteLockFile()
os.Exit(0)
l.deleteLockFile()
os.Exit(0)
})
// Check for a path.lock file
+2 -2
View File
@@ -18,7 +18,7 @@ import (
)
const (
baseStoreURL = "https://login.cloudflarewarp.com/"
baseStoreURL = "https://login.argotunnel.com/"
clientTimeout = time.Second * 60
)
@@ -45,7 +45,7 @@ func Run(transferURL *url.URL, resourceName, key, value, path string, shouldEncr
if err != nil {
fmt.Fprintf(os.Stderr, "Please open the following URL and log in with your Cloudflare account:\n\n%s\n\nLeave cloudflared running to download the %s automatically.\n", requestURL, resourceName)
} else {
fmt.Fprintf(os.Stderr, "A browser window should have opened at the following URL:\n\n%s\n\nIf the browser failed to open, open it yourself and visit the URL above.\n", requestURL)
fmt.Fprintf(os.Stderr, "A browser window should have opened at the following URL:\n\n%s\n\nIf the browser failed to open, please visit the URL above directly in your browser.\n", requestURL)
}
var resourceData []byte
+102 -71
View File
@@ -7,18 +7,18 @@ import (
"net"
"net/url"
"os"
"reflect"
"runtime"
"runtime/trace"
"sync"
"syscall"
"time"
"github.com/cloudflare/cloudflared/awsuploader"
"github.com/cloudflare/cloudflared/cmd/cloudflared/buildinfo"
"github.com/cloudflare/cloudflared/cmd/cloudflared/config"
"github.com/cloudflare/cloudflared/cmd/cloudflared/updater"
"github.com/cloudflare/cloudflared/cmd/sqlgateway"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/dbconnect"
"github.com/cloudflare/cloudflared/hello"
"github.com/cloudflare/cloudflared/metrics"
"github.com/cloudflare/cloudflared/origin"
@@ -37,7 +37,6 @@ import (
"github.com/gliderlabs/ssh"
"github.com/google/uuid"
"github.com/pkg/errors"
"golang.org/x/crypto/ssh/terminal"
"gopkg.in/urfave/cli.v2"
"gopkg.in/urfave/cli.v2/altsrc"
)
@@ -45,7 +44,7 @@ import (
const (
sentryDSN = "https://56a9c9fa5c364ab28f34b14f35ea0f1b:3e8827f6f9f740738eb11138f7bebb68@sentry.io/189878"
sshLogFileDirectory = "/var/log/cloudflared/"
sshLogFileDirectory = "/usr/local/var/log/cloudflared/"
// sshPortFlag is the port on localhost the cloudflared ssh server will run on
sshPortFlag = "local-ssh-port"
@@ -73,6 +72,11 @@ const (
// s3URLFlag is the S3 URL of SSH log uploader (e.g. don't use AWS s3 and use google storage bucket instead)
s3URLFlag = "s3-url-host"
// hostKeyPath is the path of the dir to save SSH host keys too
hostKeyPath = "host-key-path"
noIntentMsg = "The --intent argument is required. Cloudflared looks up an Intent to determine what configuration to use (i.e. which tunnels to start). If you don't have any Intents yet, you can use a placeholder Intent Label for now. Then, when you make an Intent with that label, cloudflared will get notified and open the tunnels you specified in that Intent."
)
var (
@@ -133,43 +137,7 @@ func Commands() []*cli.Command {
ArgsUsage: " ", // can't be the empty string or we get the default output
Hidden: false,
},
{
Name: "db",
Action: func(c *cli.Context) error {
tags := make(map[string]string)
tags["hostname"] = c.String("hostname")
raven.SetTagsContext(tags)
fmt.Printf("\nSQL Database Password: ")
pass, err := terminal.ReadPassword(int(syscall.Stdin))
if err != nil {
logger.Error(err)
}
go sqlgateway.StartProxy(c, logger, string(pass))
raven.CapturePanic(func() { err = tunnel(c) }, nil)
if err != nil {
raven.CaptureError(err, nil)
}
return err
},
Before: Before,
Usage: "SQL Gateway is an SQL over HTTP reverse proxy",
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "db",
Value: true,
Usage: "Enable the SQL Gateway Proxy",
},
&cli.StringFlag{
Name: "address",
Value: "",
Usage: "Database connection string: db://user:pass",
},
},
Hidden: true,
},
dbConnectCmd(),
}
var subcommands []*cli.Command
@@ -363,42 +331,53 @@ func StartServer(c *cli.Context, version string, shutdownC, graceShutdownC chan
if c.IsSet("ssh-server") {
if runtime.GOOS != "darwin" && runtime.GOOS != "linux" {
logger.Errorf("--ssh-server is not supported on %s", runtime.GOOS)
return errors.New(fmt.Sprintf("--ssh-server is not supported on %s", runtime.GOOS))
msg := fmt.Sprintf("--ssh-server is not supported on %s", runtime.GOOS)
logger.Error(msg)
return errors.New(msg)
}
logger.Infof("ssh-server set")
logManager := sshlog.NewEmptyManager()
if c.IsSet(bucketNameFlag) && c.IsSet(regionNameFlag) && c.IsSet(accessKeyIDFlag) && c.IsSet(secretIDFlag) {
uploader, err := awsuploader.NewFileUploader(c.String(bucketNameFlag), c.String(regionNameFlag),
c.String(accessKeyIDFlag), c.String(secretIDFlag), c.String(sessionTokenIDFlag), c.String(s3URLFlag))
if err != nil {
logger.WithError(err).Error("Cannot create uploader for SSH Server")
return errors.Wrap(err, "Cannot create uploader for SSH Server")
msg := "Cannot create uploader for SSH Server"
logger.WithError(err).Error(msg)
return errors.Wrap(err, msg)
}
os.Mkdir(sshLogFileDirectory, 0600)
if err := os.MkdirAll(sshLogFileDirectory, 0700); err != nil {
msg := fmt.Sprintf("Cannot create SSH log file directory %s", sshLogFileDirectory)
logger.WithError(err).Errorf(msg)
return errors.Wrap(err, msg)
}
logManager = sshlog.New(sshLogFileDirectory)
uploadManager := awsuploader.NewDirectoryUploadManager(logger, uploader, sshLogFileDirectory, 30*time.Minute, shutdownC)
uploadManager.Start()
}
logManager := sshlog.New()
sshServerAddress := "127.0.0.1:" + c.String(sshPortFlag)
server, err := sshserver.New(logManager, logger, sshServerAddress, shutdownC, c.Duration(sshIdleTimeoutFlag), c.Duration(sshMaxTimeoutFlag))
localServerAddress := "127.0.0.1:" + c.String(sshPortFlag)
server, err := sshserver.New(logManager, logger, version, localServerAddress, c.String("hostname"), c.Path(hostKeyPath), shutdownC, c.Duration(sshIdleTimeoutFlag), c.Duration(sshMaxTimeoutFlag))
if err != nil {
logger.WithError(err).Error("Cannot create new SSH Server")
return errors.Wrap(err, "Cannot create new SSH Server")
msg := "Cannot create new SSH Server"
logger.WithError(err).Error(msg)
return errors.Wrap(err, msg)
}
wg.Add(1)
go func() {
defer wg.Done()
if err = server.Start(); err != nil && err != ssh.ErrServerClosed {
logger.WithError(err).Error("SSH server error")
// TODO: remove when declarative tunnels are implemented.
close(shutdownC)
}
}()
c.Set("url", "ssh://"+sshServerAddress)
c.Set("url", "ssh://"+localServerAddress)
}
if host := hostnameFromURI(c.String("url")); host != "" {
@@ -506,19 +485,16 @@ func startDeclarativeTunnel(ctx context.Context,
return err
}
name := c.String("name")
group := c.String("group")
if group == "" {
err := fmt.Errorf("--group must be specified")
logger.WithError(err).Error("unable to parse group name")
return err
intentLabel := c.String("intent")
if intentLabel == "" {
logger.Error("--intent was empty")
return fmt.Errorf(noIntentMsg)
}
cloudflaredConfig := &connection.CloudflaredConfig{
BuildInfo: buildInfo,
CloudflaredID: cloudflaredID,
Name: name,
Group: group,
IntentLabel: intentLabel,
Tags: tags,
}
@@ -631,6 +607,60 @@ func addPortIfMissing(uri *url.URL, port int) string {
return fmt.Sprintf("%s:%d", uri.Hostname(), port)
}
func dbConnectCmd() *cli.Command {
cmd := dbconnect.Cmd()
// Append the tunnel commands so users can customize the daemon settings.
cmd.Flags = appendFlags(Flags(), cmd.Flags...)
// Override before to run tunnel validation before dbconnect validation.
cmd.Before = func(c *cli.Context) error {
err := Before(c)
if err == nil {
err = dbconnect.CmdBefore(c)
}
return err
}
// Override action to setup the Proxy, then if successful, start the tunnel daemon.
cmd.Action = func(c *cli.Context) error {
err := dbconnect.CmdAction(c)
if err == nil {
err = tunnel(c)
}
return err
}
return cmd
}
// appendFlags will append extra flags to a slice of flags.
//
// The cli package will panic if two flags exist with the same name,
// so if extraFlags contains a flag that was already defined, modify the
// original flags to use the extra version.
func appendFlags(flags []cli.Flag, extraFlags ...cli.Flag) []cli.Flag {
for _, extra := range extraFlags {
var found bool
// Check if an extra flag overrides an existing flag.
for i, flag := range flags {
if reflect.DeepEqual(extra.Names(), flag.Names()) {
flags[i] = extra
found = true
break
}
}
// Append the extra flag if it has nothing to override.
if !found {
flags = append(flags, extra)
}
}
return flags
}
func tunnelFlags(shouldHide bool) []cli.Flag {
return []cli.Flag{
&cli.StringFlag{
@@ -942,15 +972,9 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
Hidden: true,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "name",
Usage: "Friendly name for this cloudflared instance.",
EnvVars: []string{"TUNNEL_DECLARATIVE_NAME"},
Hidden: true,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "group",
Usage: "The group whose behavior this cloudflared instance will adopt. This behavior can be configured by editing the group's 'intent' in the Declarative Tunnel UI.",
EnvVars: []string{"TUNNEL_DECLARATIVE_GROUP"},
Name: "intent",
Usage: "The label of an Intent from which `cloudflared` should gets its tunnels from. Intents can be created in the Origin Registry UI.",
EnvVars: []string{"TUNNEL_INTENT"},
Hidden: true,
}),
altsrc.NewDurationFlag(&cli.DurationFlag{
@@ -963,7 +987,7 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
altsrc.NewStringFlag(&cli.StringFlag{
Name: sshPortFlag,
Usage: "Localhost port that cloudflared SSH server will run on",
Value: "22",
Value: "2222",
EnvVars: []string{"LOCAL_SSH_PORT"},
Hidden: true,
}),
@@ -1015,5 +1039,12 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
EnvVars: []string{"S3_URL"},
Hidden: true,
}),
altsrc.NewPathFlag(&cli.PathFlag{
Name: hostKeyPath,
Usage: "Absolute path of directory to save SSH host keys in",
EnvVars: []string{"HOST_KEY_PATH"},
Hidden: true,
}),
}
}
}
+1 -1
View File
@@ -15,7 +15,7 @@ import (
const (
baseLoginURL = "https://dash.cloudflare.com/argotunnel"
callbackStoreURL = "https://login.cloudflarewarp.com/"
callbackStoreURL = "https://login.argotunnel.com/"
)
func login(c *cli.Context) error {
-148
View File
@@ -1,148 +0,0 @@
package sqlgateway
import (
"database/sql"
"encoding/json"
"fmt"
"math/rand"
"net/http"
"strings"
"time"
_ "github.com/lib/pq"
cli "gopkg.in/urfave/cli.v2"
"github.com/elgs/gosqljson"
"github.com/gorilla/mux"
"github.com/sirupsen/logrus"
)
type Message struct {
Connection Connection `json:"connection"`
Command string `json:"command"`
Params []interface{} `json:"params"`
}
type Connection struct {
SSLMode string `json:"sslmode"`
Token string `json:"token"`
}
type Response struct {
Columns []string `json:"columns"`
Rows [][]string `json:"rows"`
Error string `json:"error"`
}
type Proxy struct {
Context *cli.Context
Router *mux.Router
Token string
User string
Password string
Driver string
Database string
Logger *logrus.Logger
}
func StartProxy(c *cli.Context, logger *logrus.Logger, password string) error {
proxy := NewProxy(c, logger, password)
logger.Infof("Starting SQL Gateway Proxy on port %s", strings.Split(c.String("url"), ":")[1])
err := http.ListenAndServe(":"+strings.Split(c.String("url"), ":")[1], proxy.Router)
if err != nil {
return err
}
return nil
}
func randID(n int, c *cli.Context) string {
charBytes := []byte("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890")
b := make([]byte, n)
for i := range b {
b[i] = charBytes[rand.Intn(len(charBytes))]
}
return fmt.Sprintf("%s&%s", c.String("hostname"), b)
}
// db://user@dbname
func parseInfo(input string) (string, string, string) {
p1 := strings.Split(input, "://")
p2 := strings.Split(p1[1], "@")
return p1[0], p2[0], p2[1]
}
func NewProxy(c *cli.Context, logger *logrus.Logger, pass string) *Proxy {
rand.Seed(time.Now().UnixNano())
driver, user, dbname := parseInfo(c.String("address"))
proxy := Proxy{
Context: c,
Router: mux.NewRouter(),
Token: randID(64, c),
Logger: logger,
User: user,
Password: pass,
Database: dbname,
Driver: driver,
}
logger.Info(fmt.Sprintf(`
--------------------
SQL Gateway Proxy
Token: %s
--------------------
`, proxy.Token))
proxy.Router.HandleFunc("/", proxy.proxyRequest).Methods("POST")
return &proxy
}
func (proxy *Proxy) proxyRequest(rw http.ResponseWriter, req *http.Request) {
var message Message
response := Response{}
err := json.NewDecoder(req.Body).Decode(&message)
if err != nil {
proxy.Logger.Error(err)
http.Error(rw, fmt.Sprintf("400 - %s", err.Error()), http.StatusBadRequest)
return
}
if message.Connection.Token != proxy.Token {
proxy.Logger.Error("Invalid token")
http.Error(rw, "400 - Invalid token", http.StatusBadRequest)
return
}
connStr := fmt.Sprintf("user=%s password=%s dbname=%s sslmode=%s", proxy.User, proxy.Password, proxy.Database, message.Connection.SSLMode)
db, err := sql.Open(proxy.Driver, connStr)
defer db.Close()
if err != nil {
proxy.Logger.Error(err)
http.Error(rw, fmt.Sprintf("400 - %s", err.Error()), http.StatusBadRequest)
return
} else {
proxy.Logger.Info("Forwarding SQL: ", message.Command)
rw.Header().Set("Content-Type", "application/json")
headers, data, err := gosqljson.QueryDbToArray(db, "lower", message.Command, message.Params...)
if err != nil {
proxy.Logger.Error(err)
http.Error(rw, fmt.Sprintf("400 - %s", err.Error()), http.StatusBadRequest)
return
} else {
response = Response{headers, data, ""}
}
}
json.NewEncoder(rw).Encode(response)
}
+1 -1
View File
@@ -50,7 +50,7 @@ func (c *Connection) Serve(ctx context.Context) error {
}
// Connect is used to establish connections with cloudflare's edge network
func (c *Connection) Connect(ctx context.Context, parameters *tunnelpogs.ConnectParameters, logger *logrus.Entry) (*pogs.ConnectResult, error) {
func (c *Connection) Connect(ctx context.Context, parameters *tunnelpogs.ConnectParameters, logger *logrus.Entry) (pogs.ConnectResult, error) {
openStreamCtx, cancel := context.WithTimeout(ctx, openStreamTimeout)
defer cancel()
+8 -8
View File
@@ -13,13 +13,13 @@ import (
)
const (
// Used to discover HA Warp servers
srvService = "warp"
// Used to discover HA origintunneld servers
srvService = "origintunneld"
srvProto = "tcp"
srvName = "cloudflarewarp.com"
srvName = "argotunnel.com"
// Used to fallback to DoT when we can't use the default resolver to
// discover HA Warp servers (GitHub issue #75).
// discover HA origintunneld servers (GitHub issue #75).
dotServerName = "cloudflare-dns.com"
dotServerAddr = "1.1.1.1:853"
dotTimeout = time.Duration(15 * time.Second)
@@ -30,8 +30,8 @@ const (
var friendlyDNSErrorLines = []string{
`Please try the following things to diagnose this issue:`,
` 1. ensure that cloudflarewarp.com is returning "warp" service records.`,
` Run your system's equivalent of: dig srv _warp._tcp.cloudflarewarp.com`,
` 1. ensure that argotunnel.com is returning "origintunneld" service records.`,
` Run your system's equivalent of: dig srv _origintunneld._tcp.argotunnel.com`,
` 2. ensure that your DNS resolver is not returning compressed SRV records.`,
` See GitHub issue https://github.com/golang/go/issues/27546`,
` For example, you could use Cloudflare's 1.1.1.1 as your resolver:`,
@@ -102,7 +102,7 @@ func EdgeDiscovery(logger *logrus.Entry) ([]*net.TCPAddr, error) {
// Try to fall back to DoT from Cloudflare directly.
//
// Note: Instead of DoT, we could also have used DoH. Either of these:
// - directly via the JSON API (https://1.1.1.1/dns-query?ct=application/dns-json&name=_warp._tcp.cloudflarewarp.com&type=srv)
// - directly via the JSON API (https://1.1.1.1/dns-query?ct=application/dns-json&name=_origintunneld._tcp.argotunnel.com&type=srv)
// - indirectly via `tunneldns.NewUpstreamHTTPS()`
// But both of these cases miss out on a key feature from the stdlib:
// "The returned records are sorted by priority and randomized by weight within a priority."
@@ -119,7 +119,7 @@ func EdgeDiscovery(logger *logrus.Entry) ([]*net.TCPAddr, error) {
for _, s := range friendlyDNSErrorLines {
logger.Errorln(s)
}
return nil, errors.Wrap(err, "Could not lookup srv records on _warp._tcp.cloudflarewarp.com")
return nil, errors.Wrap(err, "Could not lookup srv records on _origintunneld._tcp.argotunnel.com")
}
// Accept the fallback results and keep going
addrs = fallbackAddrs
+54 -23
View File
@@ -10,21 +10,27 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/buildinfo"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/streamhandler"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/prometheus/client_golang/prometheus"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
const (
quickStartLink = "https://developers.cloudflare.com/argo-tunnel/quickstart/"
faqLink = "https://developers.cloudflare.com/argo-tunnel/faq/"
quickStartLink = "https://developers.cloudflare.com/argo-tunnel/quickstart/"
faqLink = "https://developers.cloudflare.com/argo-tunnel/faq/"
defaultRetryAfter = time.Second * 5
packageNamespace = "connection"
edgeManagerSubsystem = "edgemanager"
)
// EdgeManager manages connections with the edge
type EdgeManager struct {
// streamHandler handles stream opened by the edge
streamHandler h2mux.MuxedStreamHandler
streamHandler *streamhandler.StreamHandler
// TLSConfig is the TLS configuration to connect with edge
tlsConfig *tls.Config
// cloudflaredConfig is the cloudflared configuration that is determined when the process first starts
@@ -35,9 +41,22 @@ type EdgeManager struct {
state *edgeManagerState
logger *logrus.Entry
metrics *metrics
}
// EdgeConnectionManagerConfigurable is the configurable attributes of a EdgeConnectionManager
type metrics struct {
// activeStreams is a gauge shared by all muxers of this process to expose the total number of active streams
activeStreams prometheus.Gauge
}
func newMetrics(namespace, subsystem string) *metrics {
return &metrics{
activeStreams: h2mux.NewActiveStreamsMetrics(namespace, subsystem),
}
}
// EdgeManagerConfigurable is the configurable attributes of a EdgeConnectionManager
type EdgeManagerConfigurable struct {
TunnelHostnames []h2mux.TunnelHostname
*pogs.EdgeConnectionConfig
@@ -47,12 +66,11 @@ type CloudflaredConfig struct {
CloudflaredID uuid.UUID
Tags []pogs.Tag
BuildInfo *buildinfo.BuildInfo
Name string
Group string
IntentLabel string
}
func NewEdgeManager(
streamHandler h2mux.MuxedStreamHandler,
streamHandler *streamhandler.StreamHandler,
edgeConnMgrConfigurable *EdgeManagerConfigurable,
userCredential []byte,
tlsConfig *tls.Config,
@@ -67,6 +85,7 @@ func NewEdgeManager(
serviceDiscoverer: serviceDiscoverer,
state: newEdgeConnectionManagerState(edgeConnMgrConfigurable, userCredential),
logger: logger.WithField("subsystem", "connectionManager"),
metrics: newMetrics(packageNamespace, edgeManagerSubsystem),
}
}
@@ -88,8 +107,12 @@ func (em *EdgeManager) Run(ctx context.Context) error {
// Create/delete connection one at a time, so we don't need to adjust for connections that are being created/deleted
// in shouldCreateConnection or shouldReduceConnection calculation
if em.state.shouldCreateConnection(em.serviceDiscoverer.AvailableAddrs()) {
if err := em.newConnection(ctx); err != nil {
em.logger.WithError(err).Error("cannot create new connection")
if connErr := em.newConnection(ctx); connErr != nil {
if !connErr.ShouldRetry {
em.logger.WithError(connErr).Error(em.noRetryMessage())
return connErr
}
em.logger.WithError(connErr).Error("cannot create new connection")
}
} else if em.state.shouldReduceConnection() {
if err := em.closeConnection(ctx); err != nil {
@@ -104,11 +127,11 @@ func (em *EdgeManager) UpdateConfigurable(newConfigurable *EdgeManagerConfigurab
em.state.updateConfigurable(newConfigurable)
}
func (em *EdgeManager) newConnection(ctx context.Context) error {
func (em *EdgeManager) newConnection(ctx context.Context) *pogs.ConnectError {
edgeIP := em.serviceDiscoverer.Addr()
edgeConn, err := em.dialEdge(ctx, edgeIP)
if err != nil {
return errors.Wrap(err, "dial edge error")
return retryConnection(fmt.Sprintf("dial edge error: %v", err))
}
configurable := em.state.getConfigurable()
// Establish a muxed connection with the edge
@@ -120,14 +143,14 @@ func (em *EdgeManager) newConnection(ctx context.Context) error {
HeartbeatInterval: configurable.HeartbeatInterval,
MaxHeartbeats: configurable.MaxFailedHeartbeats,
Logger: em.logger.WithField("subsystem", "muxer"),
})
}, em.metrics.activeStreams)
if err != nil {
return errors.Wrap(err, "couldn't perform handshake with edge")
retryConnection(fmt.Sprintf("couldn't perform handshake with edge: %v", err))
}
h2muxConn, err := newConnection(muxer, edgeIP)
if err != nil {
return errors.Wrap(err, "couldn't create h2mux connection")
return retryConnection(fmt.Sprintf("couldn't create h2mux connection: %v", err))
}
go em.serveConn(ctx, h2muxConn)
@@ -137,24 +160,24 @@ func (em *EdgeManager) newConnection(ctx context.Context) error {
CloudflaredVersion: em.cloudflaredConfig.BuildInfo.CloudflaredVersion,
NumPreviousAttempts: 0,
OriginCert: em.state.getUserCredential(),
Name: em.cloudflaredConfig.Name,
Group: em.cloudflaredConfig.Group,
IntentLabel: em.cloudflaredConfig.IntentLabel,
Tags: em.cloudflaredConfig.Tags,
}, em.logger)
if err != nil {
h2muxConn.Shutdown()
return errors.Wrap(err, "couldn't connect to edge")
return retryConnection(fmt.Sprintf("couldn't connect to edge: %v", err))
}
if connErr := connResult.Err; connErr != nil {
if !connErr.ShouldRetry {
return errors.Wrap(connErr, em.noRetryMessage())
}
return errors.Wrapf(connErr, "edge responded with RetryAfter=%v", connErr.RetryAfter)
if connErr := connResult.ConnectError(); connErr != nil {
return connErr
}
em.state.newConnection(h2muxConn)
em.logger.Infof("connected to %s", connResult.ServerInfo.LocationName)
em.logger.Infof("connected to %s", connResult.ConnectedTo())
if connResult.ClientConfig() != nil {
em.streamHandler.UseConfiguration(ctx, connResult.ClientConfig())
}
return nil
}
@@ -284,3 +307,11 @@ func (ems *edgeManagerState) getUserCredential() []byte {
defer ems.RUnlock()
return ems.userCredential
}
func retryConnection(cause string) *pogs.ConnectError {
return &pogs.ConnectError{
Cause: cause,
RetryAfter: defaultRetryAfter,
ShouldRetry: true,
}
}
+6 -8
View File
@@ -8,7 +8,9 @@ import (
"github.com/stretchr/testify/assert"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/streamhandler"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/google/uuid"
"github.com/sirupsen/logrus"
)
@@ -42,16 +44,12 @@ var (
}
)
type mockStreamHandler struct {
}
func (msh *mockStreamHandler) ServeStream(*h2mux.MuxedStream) error {
return nil
}
func mockEdgeManager() *EdgeManager {
newConfigChan := make(chan<- *pogs.ClientConfig)
useConfigResultChan := make(<-chan *pogs.UseConfigurationResult)
logger := logrus.New()
return NewEdgeManager(
&mockStreamHandler{},
streamhandler.NewStreamHandler(newConfigChan, useConfigResultChan, logger),
configurable,
[]byte{},
nil,
+145
View File
@@ -0,0 +1,145 @@
package dbconnect
import (
"context"
"encoding/json"
"fmt"
"net/url"
"strings"
"time"
"unicode"
"unicode/utf8"
)
// Client is an interface to talk to any database.
//
// Currently, the only implementation is SQLClient, but its structure
// should be designed to handle a MongoClient or RedisClient in the future.
type Client interface {
Ping(context.Context) error
Submit(context.Context, *Command) (interface{}, error)
}
// NewClient creates a database client based on its URL scheme.
func NewClient(ctx context.Context, originURL *url.URL) (Client, error) {
return NewSQLClient(ctx, originURL)
}
// Command is a standard, non-vendor format for submitting database commands.
//
// When determining the scope of this struct, refer to the following litmus test:
// Could this (roughly) conform to SQL, Document-based, and Key-value command formats?
type Command struct {
Statement string `json:"statement"`
Arguments Arguments `json:"arguments,omitempty"`
Mode string `json:"mode,omitempty"`
Isolation string `json:"isolation,omitempty"`
Timeout time.Duration `json:"timeout,omitempty"`
}
// Validate enforces the contract of Command: non empty statement (both in length and logic),
// lowercase mode and isolation, non-zero timeout, and valid Arguments.
func (cmd *Command) Validate() error {
if cmd.Statement == "" {
return fmt.Errorf("cannot provide an empty statement")
}
if strings.Map(func(char rune) rune {
if char == ';' || unicode.IsSpace(char) {
return -1
}
return char
}, cmd.Statement) == "" {
return fmt.Errorf("cannot provide a statement with no logic: '%s'", cmd.Statement)
}
cmd.Mode = strings.ToLower(cmd.Mode)
cmd.Isolation = strings.ToLower(cmd.Isolation)
if cmd.Timeout.Nanoseconds() <= 0 {
cmd.Timeout = 24 * time.Hour
}
return cmd.Arguments.Validate()
}
// UnmarshalJSON converts a byte representation of JSON into a Command, which is also validated.
func (cmd *Command) UnmarshalJSON(data []byte) error {
// Alias is required to avoid infinite recursion from the default UnmarshalJSON.
type Alias Command
alias := &struct {
*Alias
}{
Alias: (*Alias)(cmd),
}
err := json.Unmarshal(data, &alias)
if err == nil {
err = cmd.Validate()
}
return err
}
// Arguments is a wrapper for either map-based or array-based Command arguments.
//
// Each field is mutually-exclusive and some Client implementations may not
// support both fields (eg. MySQL does not accept named arguments).
type Arguments struct {
Named map[string]interface{}
Positional []interface{}
}
// Validate enforces the contract of Arguments: non nil, mutually exclusive, and no empty or reserved keys.
func (args *Arguments) Validate() error {
if args.Named == nil {
args.Named = map[string]interface{}{}
}
if args.Positional == nil {
args.Positional = []interface{}{}
}
if len(args.Named) > 0 && len(args.Positional) > 0 {
return fmt.Errorf("both named and positional arguments cannot be specified: %+v and %+v", args.Named, args.Positional)
}
for key := range args.Named {
if key == "" {
return fmt.Errorf("named arguments cannot contain an empty key: %+v", args.Named)
}
if !utf8.ValidString(key) {
return fmt.Errorf("named argument does not conform to UTF-8 encoding: %s", key)
}
if strings.HasPrefix(key, "_") {
return fmt.Errorf("named argument cannot start with a reserved keyword '_': %s", key)
}
if unicode.IsNumber([]rune(key)[0]) {
return fmt.Errorf("named argument cannot start with a number: %s", key)
}
}
return nil
}
// UnmarshalJSON converts a byte representation of JSON into Arguments, which is also validated.
func (args *Arguments) UnmarshalJSON(data []byte) error {
var obj interface{}
err := json.Unmarshal(data, &obj)
if err != nil {
return err
}
named, ok := obj.(map[string]interface{})
if ok {
args.Named = named
} else {
positional, ok := obj.([]interface{})
if ok {
args.Positional = positional
} else {
return fmt.Errorf("arguments must either be an object {\"0\":\"val\"} or an array [\"val\"]: %s", string(data))
}
}
return args.Validate()
}
+183
View File
@@ -0,0 +1,183 @@
package dbconnect
import (
"encoding/json"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestCommandValidateEmpty(t *testing.T) {
stmts := []string{
"",
";",
" \n\t",
";\n;\t;",
}
for _, stmt := range stmts {
cmd := Command{Statement: stmt}
assert.Error(t, cmd.Validate(), stmt)
}
}
func TestCommandValidateMode(t *testing.T) {
modes := []string{
"",
"query",
"ExEc",
"PREPARE",
}
for _, mode := range modes {
cmd := Command{Statement: "Ok", Mode: mode}
assert.NoError(t, cmd.Validate(), mode)
assert.Equal(t, strings.ToLower(mode), cmd.Mode)
}
}
func TestCommandValidateIsolation(t *testing.T) {
isos := []string{
"",
"default",
"read_committed",
"SNAPshot",
}
for _, iso := range isos {
cmd := Command{Statement: "Ok", Isolation: iso}
assert.NoError(t, cmd.Validate(), iso)
assert.Equal(t, strings.ToLower(iso), cmd.Isolation)
}
}
func TestCommandValidateTimeout(t *testing.T) {
cmd := Command{Statement: "Ok", Timeout: 0}
assert.NoError(t, cmd.Validate())
assert.NotZero(t, cmd.Timeout)
cmd = Command{Statement: "Ok", Timeout: 1 * time.Second}
assert.NoError(t, cmd.Validate())
assert.Equal(t, 1*time.Second, cmd.Timeout)
}
func TestCommandValidateArguments(t *testing.T) {
cmd := Command{Statement: "Ok", Arguments: Arguments{
Named: map[string]interface{}{"key": "val"},
Positional: []interface{}{"val"},
}}
assert.Error(t, cmd.Validate())
}
func TestCommandUnmarshalJSON(t *testing.T) {
strs := []string{
"{\"statement\":\"Ok\"}",
"{\"statement\":\"Ok\",\"arguments\":[0, 3.14, \"apple\"],\"mode\":\"query\"}",
"{\"statement\":\"Ok\",\"isolation\":\"read_uncommitted\",\"timeout\":1000}",
}
for _, str := range strs {
var cmd Command
assert.NoError(t, json.Unmarshal([]byte(str), &cmd), str)
}
strs = []string{
"",
"\"",
"{}",
"{\"argument\":{\"key\":\"val\"}}",
"{\"statement\":[\"Ok\"]}",
}
for _, str := range strs {
var cmd Command
assert.Error(t, json.Unmarshal([]byte(str), &cmd), str)
}
}
func TestArgumentsValidateNotNil(t *testing.T) {
args := Arguments{}
assert.NoError(t, args.Validate())
assert.NotNil(t, args.Named)
assert.NotNil(t, args.Positional)
}
func TestArgumentsValidateMutuallyExclusive(t *testing.T) {
args := []Arguments{
Arguments{},
Arguments{Named: map[string]interface{}{"key": "val"}},
Arguments{Positional: []interface{}{"val"}},
}
for _, arg := range args {
assert.NoError(t, arg.Validate())
assert.False(t, len(arg.Named) > 0 && len(arg.Positional) > 0)
}
args = []Arguments{
Arguments{
Named: map[string]interface{}{"key": "val"},
Positional: []interface{}{"val"},
},
}
for _, arg := range args {
assert.Error(t, arg.Validate())
assert.True(t, len(arg.Named) > 0 && len(arg.Positional) > 0)
}
}
func TestArgumentsValidateKeys(t *testing.T) {
keys := []string{
"",
"_",
"_key",
"1",
"1key",
"\xf0\x28\x8c\xbc", // non-utf8
}
for _, key := range keys {
args := Arguments{Named: map[string]interface{}{key: "val"}}
assert.Error(t, args.Validate(), key)
}
}
func TestArgumentsUnmarshalJSON(t *testing.T) {
strs := []string{
"{}",
"{\"key\":\"val\"}",
"{\"key\":[1, 3.14, {\"key\":\"val\"}]}",
"[]",
"[\"key\",\"val\"]",
"[{}]",
}
for _, str := range strs {
var args Arguments
assert.NoError(t, json.Unmarshal([]byte(str), &args), str)
}
strs = []string{
"",
"\"",
"1",
"\"key\"",
"{\"key\",\"val\"}",
}
for _, str := range strs {
var args Arguments
assert.Error(t, json.Unmarshal([]byte(str), &args), str)
}
}
+157
View File
@@ -0,0 +1,157 @@
package dbconnect
import (
"context"
"log"
"net"
"strconv"
"gopkg.in/urfave/cli.v2"
"gopkg.in/urfave/cli.v2/altsrc"
)
// Cmd is the entrypoint command for dbconnect.
//
// The tunnel package is responsible for appending this to tunnel.Commands().
func Cmd() *cli.Command {
return &cli.Command{
Category: "Database Connect (ALPHA)",
Name: "db-connect",
Usage: "Access your SQL database from Cloudflare Workers or the browser",
ArgsUsage: " ",
Description: `
Creates a connection between your database and the Cloudflare edge.
Now you can execute SQL commands anywhere you can send HTTPS requests.
Connect your database with any of the following commands, you can also try the "playground" without a database:
cloudflared db-connect --hostname sql.mysite.com --url postgres://user:pass@localhost?sslmode=disable \
--auth-domain mysite.cloudflareaccess.com --application-aud my-access-policy-tag
cloudflared db-connect --hostname sql-dev.mysite.com --url mysql://localhost --insecure
cloudflared db-connect --playground
Requests should be authenticated using Cloudflare Access, learn more about how to enable it here:
https://developers.cloudflare.com/access/service-auth/service-token/
`,
Flags: []cli.Flag{
altsrc.NewStringFlag(&cli.StringFlag{
Name: "url",
Usage: "URL to the database (eg. postgres://user:pass@localhost?sslmode=disable)",
EnvVars: []string{"TUNNEL_URL"},
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "hostname",
Usage: "Hostname to accept commands over HTTPS (eg. sql.mysite.com)",
EnvVars: []string{"TUNNEL_HOSTNAME"},
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "auth-domain",
Usage: "Cloudflare Access authentication domain for your account (eg. mysite.cloudflareaccess.com)",
EnvVars: []string{"TUNNEL_ACCESS_AUTH_DOMAIN"},
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "application-aud",
Usage: "Cloudflare Access application \"AUD\" to verify JWTs from requests",
EnvVars: []string{"TUNNEL_ACCESS_APPLICATION_AUD"},
}),
altsrc.NewBoolFlag(&cli.BoolFlag{
Name: "insecure",
Usage: "Disable authentication, the database will be open to the Internet",
Value: false,
EnvVars: []string{"TUNNEL_ACCESS_INSECURE"},
}),
altsrc.NewBoolFlag(&cli.BoolFlag{
Name: "playground",
Usage: "Run a temporary, in-memory SQLite3 database for testing",
Value: false,
EnvVars: []string{"TUNNEL_HELLO_WORLD"},
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "loglevel",
Value: "debug", // Make it more verbose than the tunnel default 'info'.
EnvVars: []string{"TUNNEL_LOGLEVEL"},
Hidden: true,
}),
},
Before: CmdBefore,
Action: CmdAction,
Hidden: true,
}
}
// CmdBefore runs some validation checks before running the command.
func CmdBefore(c *cli.Context) error {
// Show the help text is no flags are specified.
if c.NumFlags() == 0 {
return cli.ShowSubcommandHelp(c)
}
// Hello-world and playground are synonymous with each other,
// unset hello-world to prevent tunnel from initializing the hello package.
if c.IsSet("hello-world") {
c.Set("playground", "true")
c.Set("hello-world", "false")
}
// Unix-socket database urls are supported, but the logic is the same as url.
if c.IsSet("unix-socket") {
c.Set("url", c.String("unix-socket"))
c.Set("unix-socket", "")
}
// When playground mode is enabled, run with an in-memory database.
if c.IsSet("playground") {
c.Set("url", "sqlite3::memory:?cache=shared")
c.Set("insecure", strconv.FormatBool(!c.IsSet("auth-domain") && !c.IsSet("application-aud")))
}
// At this point, insecure configurations are valid.
if c.Bool("insecure") {
return nil
}
// Ensure that secure configurations specify a hostname, domain, and tag for JWT validation.
if !c.IsSet("hostname") || !c.IsSet("auth-domain") || !c.IsSet("application-aud") {
log.Fatal("must specify --hostname, --auth-domain, and --application-aud unless you want to run in --insecure mode")
}
return nil
}
// CmdAction starts the Proxy and sets the url in cli.Context to point to the Proxy address.
func CmdAction(c *cli.Context) error {
// STOR-612: sync with context in tunnel daemon.
ctx := context.Background()
var proxy *Proxy
var err error
if c.Bool("insecure") {
proxy, err = NewInsecureProxy(ctx, c.String("url"))
} else {
proxy, err = NewSecureProxy(ctx, c.String("url"), c.String("auth-domain"), c.String("application-aud"))
}
if err != nil {
log.Fatal(err)
return err
}
listenerC := make(chan net.Listener)
defer close(listenerC)
// Since the Proxy should only talk to the tunnel daemon, find the next available
// localhost port and start to listen to requests.
go func() {
err := proxy.Start(ctx, "127.0.0.1:", listenerC)
if err != nil {
log.Fatal(err)
}
}()
// Block until the the Proxy is online, retreive its address, and change the url to point to it.
// This is effectively "handing over" control to the tunnel package so it can run the tunnel daemon.
c.Set("url", "https://"+(<-listenerC).Addr().String())
return nil
}
+27
View File
@@ -0,0 +1,27 @@
package dbconnect
import (
"testing"
"github.com/stretchr/testify/assert"
"gopkg.in/urfave/cli.v2"
)
func TestCmd(t *testing.T) {
tests := [][]string{
{"cloudflared", "db-connect", "--playground"},
{"cloudflared", "db-connect", "--playground", "--hostname", "sql.mysite.com"},
{"cloudflared", "db-connect", "--url", "sqlite3::memory:?cache=shared", "--insecure"},
{"cloudflared", "db-connect", "--url", "sqlite3::memory:?cache=shared", "--hostname", "sql.mysite.com", "--auth-domain", "mysite.cloudflareaccess.com", "--application-aud", "aud"},
}
app := &cli.App{
Name: "cloudflared",
Commands: []*cli.Command{Cmd()},
}
for _, test := range tests {
assert.NoError(t, app.Run(test))
}
}
+271
View File
@@ -0,0 +1,271 @@
package dbconnect
import (
"context"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/url"
"time"
"github.com/cloudflare/cloudflared/hello"
"github.com/cloudflare/cloudflared/validation"
"github.com/gorilla/mux"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
// Proxy is an HTTP server that proxies requests to a Client.
type Proxy struct {
client Client
accessValidator *validation.Access
logger *logrus.Logger
}
// NewInsecureProxy creates a Proxy that talks to a Client at an origin.
//
// In insecure mode, the Proxy will allow all Command requests.
func NewInsecureProxy(ctx context.Context, origin string) (*Proxy, error) {
originURL, err := url.Parse(origin)
if err != nil {
return nil, errors.Wrap(err, "must provide a valid database url")
}
client, err := NewClient(ctx, originURL)
if err != nil {
return nil, err
}
err = client.Ping(ctx)
if err != nil {
return nil, errors.Wrap(err, "could not connect to the database")
}
return &Proxy{client, nil, logrus.New()}, nil
}
// NewSecureProxy creates a Proxy that talks to a Client at an origin.
//
// In secure mode, the Proxy will reject any Command requests that are
// not authenticated by Cloudflare Access with a valid JWT.
func NewSecureProxy(ctx context.Context, origin, authDomain, applicationAUD string) (*Proxy, error) {
proxy, err := NewInsecureProxy(ctx, origin)
if err != nil {
return nil, err
}
validator, err := validation.NewAccessValidator(ctx, authDomain, authDomain, applicationAUD)
if err != nil {
return nil, err
}
proxy.accessValidator = validator
return proxy, err
}
// IsInsecure gets whether the Proxy will accept a Command from any source.
func (proxy *Proxy) IsInsecure() bool {
return proxy.accessValidator == nil
}
// IsAllowed checks whether a http.Request is allowed to receive data.
//
// By default, requests must pass through Cloudflare Access for authentication.
// If the proxy is explcitly set to insecure mode, all requests will be allowed.
func (proxy *Proxy) IsAllowed(r *http.Request, verbose ...bool) bool {
if proxy.IsInsecure() {
return true
}
// Access and Tunnel should prevent bad JWTs from even reaching the origin,
// but validate tokens anyway as an abundance of caution.
err := proxy.accessValidator.ValidateRequest(r.Context(), r)
if err == nil {
return true
}
// Warn administrators that invalid JWTs are being rejected. This is indicative
// of either a misconfiguration of the CLI or a massive failure of upstream systems.
if len(verbose) > 0 {
proxy.httpLog(r, err).Error("Failed JWT authentication")
}
return false
}
// Start the Proxy at a given address and notify the listener channel when the server is online.
func (proxy *Proxy) Start(ctx context.Context, addr string, listenerC chan<- net.Listener) error {
// STOR-611: use a seperate listener and consider web socket support.
httpListener, err := hello.CreateTLSListener(addr)
if err != nil {
return errors.Wrapf(err, "could not create listener at %s", addr)
}
errC := make(chan error)
defer close(errC)
// Starts the HTTP server and begins to serve requests.
go func() {
errC <- proxy.httpListen(ctx, httpListener)
}()
// Continually ping the server until it comes online or 10 attempts fail.
go func() {
var err error
for i := 0; i < 10; i++ {
_, err = http.Get("http://" + httpListener.Addr().String())
// Once no error was detected, notify the listener channel and return.
if err == nil {
listenerC <- httpListener
return
}
// Backoff between requests to ping the server.
<-time.After(1 * time.Second)
}
errC <- errors.Wrap(err, "took too long for the http server to start")
}()
return <-errC
}
// httpListen starts the httpServer and blocks until the context closes.
func (proxy *Proxy) httpListen(ctx context.Context, listener net.Listener) error {
httpServer := &http.Server{
Addr: listener.Addr().String(),
Handler: proxy.httpRouter(),
ReadTimeout: 10 * time.Second,
WriteTimeout: 60 * time.Second,
IdleTimeout: 60 * time.Second,
}
go func() {
<-ctx.Done()
httpServer.Close()
listener.Close()
}()
return httpServer.Serve(listener)
}
// httpRouter creates a mux.Router for the Proxy.
func (proxy *Proxy) httpRouter() *mux.Router {
router := mux.NewRouter()
router.HandleFunc("/ping", proxy.httpPing()).Methods("GET", "HEAD")
router.HandleFunc("/submit", proxy.httpSubmit()).Methods("POST")
return router
}
// httpPing tests the connection to the database.
//
// By default, this endpoint is unauthenticated to allow for health checks.
// To enable authentication, Cloudflare Access must be enabled on this route.
func (proxy *Proxy) httpPing() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
err := proxy.client.Ping(ctx)
if err == nil {
proxy.httpRespond(w, r, http.StatusOK, "")
} else {
proxy.httpRespondErr(w, r, http.StatusInternalServerError, err)
}
}
}
// httpSubmit sends a command to the database and returns its response.
//
// By default, this endpoint will reject requests that do not pass through Cloudflare Access.
// To disable authentication, the --insecure flag must be specified in the command line.
func (proxy *Proxy) httpSubmit() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if !proxy.IsAllowed(r, true) {
proxy.httpRespondErr(w, r, http.StatusForbidden, fmt.Errorf(""))
return
}
var cmd Command
err := json.NewDecoder(r.Body).Decode(&cmd)
if err != nil {
proxy.httpRespondErr(w, r, http.StatusBadRequest, err)
return
}
ctx := r.Context()
data, err := proxy.client.Submit(ctx, &cmd)
if err != nil {
proxy.httpRespondErr(w, r, http.StatusUnprocessableEntity, err)
return
}
w.Header().Set("Content-type", "application/json")
err = json.NewEncoder(w).Encode(data)
if err != nil {
proxy.httpRespondErr(w, r, http.StatusInternalServerError, err)
}
}
}
// httpRespond writes a status code and string response to the response writer.
func (proxy *Proxy) httpRespond(w http.ResponseWriter, r *http.Request, status int, message string) {
w.WriteHeader(status)
// Only expose the message detail of the reponse if the request is not HEAD
// and the user is authenticated. For example, this prevents an unauthenticated
// failed health check from accidentally leaking sensitive information about the Client.
if r.Method != http.MethodHead && proxy.IsAllowed(r) {
if message == "" {
message = http.StatusText(status)
}
fmt.Fprint(w, message)
}
}
// httpRespondErr is similar to httpRespond, except it formats errors to be more friendly.
func (proxy *Proxy) httpRespondErr(w http.ResponseWriter, r *http.Request, defaultStatus int, err error) {
status, err := httpError(defaultStatus, err)
proxy.httpRespond(w, r, status, err.Error())
if len(err.Error()) > 0 {
proxy.httpLog(r, err).Warn("Database proxy error")
}
}
// httpLog returns a logrus.Entry that is formatted to output a request Cf-ray.
func (proxy *Proxy) httpLog(r *http.Request, err error) *logrus.Entry {
return proxy.logger.WithContext(r.Context()).WithField("CF-RAY", r.Header.Get("Cf-ray")).WithError(err)
}
// httpError extracts common errors and returns an status code and friendly error.
func httpError(defaultStatus int, err error) (int, error) {
if err == nil {
return http.StatusNotImplemented, fmt.Errorf("error expected but found none")
}
if err == io.EOF {
return http.StatusBadRequest, fmt.Errorf("request body cannot be empty")
}
if err == context.DeadlineExceeded {
return http.StatusRequestTimeout, err
}
_, ok := err.(net.Error)
if ok {
return http.StatusRequestTimeout, err
}
if err == context.Canceled {
// Does not exist in Golang, but would be: http.StatusClientClosedWithoutResponse
return 444, err
}
return defaultStatus, err
}
+238
View File
@@ -0,0 +1,238 @@
package dbconnect
import (
"context"
"fmt"
"io"
"io/ioutil"
"net"
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gorilla/mux"
"github.com/stretchr/testify/assert"
)
func TestNewInsecureProxy(t *testing.T) {
origins := []string{
"",
":/",
"http://localhost",
"tcp://localhost:9000?debug=true",
"mongodb://127.0.0.1",
}
for _, origin := range origins {
proxy, err := NewInsecureProxy(context.Background(), origin)
assert.Error(t, err)
assert.Empty(t, proxy)
}
}
func TestProxyIsAllowed(t *testing.T) {
proxy := helperNewProxy(t)
req := httptest.NewRequest("GET", "https://1.1.1.1/ping", nil)
assert.True(t, proxy.IsAllowed(req))
proxy = helperNewProxy(t, true)
req.Header.Set("Cf-access-jwt-assertion", "xxx")
assert.False(t, proxy.IsAllowed(req))
}
func TestProxyStart(t *testing.T) {
proxy := helperNewProxy(t)
ctx := context.Background()
listenerC := make(chan net.Listener)
err := proxy.Start(ctx, "1.1.1.1:", listenerC)
assert.Error(t, err)
err = proxy.Start(ctx, "127.0.0.1:-1", listenerC)
assert.Error(t, err)
ctx, cancel := context.WithTimeout(ctx, 0)
defer cancel()
err = proxy.Start(ctx, "127.0.0.1:", listenerC)
assert.IsType(t, http.ErrServerClosed, err)
}
func TestProxyHTTPRouter(t *testing.T) {
proxy := helperNewProxy(t)
router := proxy.httpRouter()
tests := []struct {
path string
method string
valid bool
}{
{"", "GET", false},
{"/", "GET", false},
{"/ping", "GET", true},
{"/ping", "HEAD", true},
{"/ping", "POST", false},
{"/submit", "POST", true},
{"/submit", "GET", false},
{"/submit/extra", "POST", false},
}
for _, test := range tests {
match := &mux.RouteMatch{}
ok := router.Match(httptest.NewRequest(test.method, "https://1.1.1.1"+test.path, nil), match)
assert.True(t, ok == test.valid, test.path)
}
}
func TestProxyHTTPPing(t *testing.T) {
proxy := helperNewProxy(t)
server := httptest.NewServer(proxy.httpPing())
defer server.Close()
client := server.Client()
res, err := client.Get(server.URL)
assert.NoError(t, err)
assert.Equal(t, http.StatusOK, res.StatusCode)
assert.Equal(t, int64(2), res.ContentLength)
res, err = client.Head(server.URL)
assert.NoError(t, err)
assert.Equal(t, http.StatusOK, res.StatusCode)
assert.Equal(t, int64(-1), res.ContentLength)
}
func TestProxyHTTPSubmit(t *testing.T) {
proxy := helperNewProxy(t)
server := httptest.NewServer(proxy.httpSubmit())
defer server.Close()
client := server.Client()
tests := []struct {
input string
status int
output string
}{
{"", http.StatusBadRequest, "request body cannot be empty"},
{"{}", http.StatusBadRequest, "cannot provide an empty statement"},
{"{\"statement\":\"Ok\"}", http.StatusUnprocessableEntity, "cannot provide invalid sql mode: ''"},
{"{\"statement\":\"Ok\",\"mode\":\"query\"}", http.StatusUnprocessableEntity, "near \"Ok\": syntax error"},
{"{\"statement\":\"CREATE TABLE t (a INT);\",\"mode\":\"exec\"}", http.StatusOK, "{\"last_insert_id\":0,\"rows_affected\":0}\n"},
}
for _, test := range tests {
res, err := client.Post(server.URL, "application/json", strings.NewReader(test.input))
assert.NoError(t, err)
assert.Equal(t, test.status, res.StatusCode)
if res.StatusCode > http.StatusOK {
assert.Equal(t, "text/plain; charset=utf-8", res.Header.Get("Content-type"))
} else {
assert.Equal(t, "application/json", res.Header.Get("Content-type"))
}
data, err := ioutil.ReadAll(res.Body)
defer res.Body.Close()
str := string(data)
assert.NoError(t, err)
assert.Equal(t, test.output, str)
}
}
func TestProxyHTTPSubmitForbidden(t *testing.T) {
proxy := helperNewProxy(t, true)
server := httptest.NewServer(proxy.httpSubmit())
defer server.Close()
client := server.Client()
res, err := client.Get(server.URL)
assert.NoError(t, err)
assert.Equal(t, http.StatusForbidden, res.StatusCode)
assert.Zero(t, res.ContentLength)
}
func TestProxyHTTPRespond(t *testing.T) {
proxy := helperNewProxy(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
proxy.httpRespond(w, r, http.StatusAccepted, "Hello")
}))
defer server.Close()
client := server.Client()
res, err := client.Get(server.URL)
assert.NoError(t, err)
assert.Equal(t, http.StatusAccepted, res.StatusCode)
assert.Equal(t, int64(5), res.ContentLength)
data, err := ioutil.ReadAll(res.Body)
defer res.Body.Close()
assert.Equal(t, []byte("Hello"), data)
}
func TestProxyHTTPRespondForbidden(t *testing.T) {
proxy := helperNewProxy(t, true)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
proxy.httpRespond(w, r, http.StatusAccepted, "Hello")
}))
defer server.Close()
client := server.Client()
res, err := client.Get(server.URL)
assert.NoError(t, err)
assert.Equal(t, http.StatusAccepted, res.StatusCode)
assert.Equal(t, int64(0), res.ContentLength)
}
func TestHTTPError(t *testing.T) {
_, errTimeout := net.DialTimeout("tcp", "127.0.0.1", 0)
assert.Error(t, errTimeout)
tests := []struct {
input error
status int
output error
}{
{nil, http.StatusNotImplemented, fmt.Errorf("error expected but found none")},
{io.EOF, http.StatusBadRequest, fmt.Errorf("request body cannot be empty")},
{context.DeadlineExceeded, http.StatusRequestTimeout, nil},
{context.Canceled, 444, nil},
{errTimeout, http.StatusRequestTimeout, nil},
{fmt.Errorf(""), http.StatusInternalServerError, nil},
}
for _, test := range tests {
status, err := httpError(http.StatusInternalServerError, test.input)
assert.Error(t, err)
assert.Equal(t, test.status, status)
if test.output == nil {
test.output = test.input
}
assert.Equal(t, test.output, err)
}
}
func helperNewProxy(t *testing.T, secure ...bool) *Proxy {
t.Helper()
proxy, err := NewSecureProxy(context.Background(), "file::memory:?cache=shared", "test.cloudflareaccess.com", "")
assert.NoError(t, err)
assert.NotNil(t, proxy)
if len(secure) == 0 {
proxy.accessValidator = nil // Mark as insecure
}
return proxy
}
+318
View File
@@ -0,0 +1,318 @@
package dbconnect
import (
"context"
"database/sql"
"encoding/json"
"fmt"
"net/url"
"reflect"
"strings"
"github.com/jmoiron/sqlx"
"github.com/pkg/errors"
"github.com/xo/dburl"
// SQL drivers self-register with the database/sql package.
// https://github.com/golang/go/wiki/SQLDrivers
_ "github.com/denisenkom/go-mssqldb"
_ "github.com/go-sql-driver/mysql"
_ "github.com/mattn/go-sqlite3"
"github.com/kshvakov/clickhouse"
"github.com/lib/pq"
)
// SQLClient is a Client that talks to a SQL database.
type SQLClient struct {
Dialect string
driver *sqlx.DB
}
// NewSQLClient creates a SQL client based on its URL scheme.
func NewSQLClient(ctx context.Context, originURL *url.URL) (Client, error) {
res, err := dburl.Parse(originURL.String())
if err != nil {
helpText := fmt.Sprintf("supported drivers: %+q, see documentation for more details: %s", sql.Drivers(), "https://godoc.org/github.com/xo/dburl")
return nil, fmt.Errorf("could not parse sql database url '%s': %s\n%s", originURL, err.Error(), helpText)
}
// Establishes the driver, but does not test the connection.
driver, err := sqlx.Open(res.Driver, res.DSN)
if err != nil {
return nil, fmt.Errorf("could not open sql driver %s: %s\n%s", res.Driver, err.Error(), res.DSN)
}
// Closes the driver, will occur when the context finishes.
go func() {
<-ctx.Done()
driver.Close()
}()
return &SQLClient{driver.DriverName(), driver}, nil
}
// Ping verifies a connection to the database is still alive.
func (client *SQLClient) Ping(ctx context.Context) error {
return client.driver.PingContext(ctx)
}
// Submit queries or executes a command to the SQL database.
func (client *SQLClient) Submit(ctx context.Context, cmd *Command) (interface{}, error) {
txx, err := cmd.ValidateSQL(client.Dialect)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(ctx, cmd.Timeout)
defer cancel()
var res interface{}
// Get the next available sql.Conn and submit the Command.
err = sqlConn(ctx, client.driver, txx, func(conn *sql.Conn) error {
stmt := cmd.Statement
args := cmd.Arguments.Positional
if cmd.Mode == "query" {
res, err = sqlQuery(ctx, conn, stmt, args)
} else {
res, err = sqlExec(ctx, conn, stmt, args)
}
return err
})
return res, err
}
// ValidateSQL extends the contract of Command for SQL dialects:
// mode is conformed, arguments are []sql.NamedArg, and isolation is a sql.IsolationLevel.
//
// When the command should not be wrapped in a transaction, *sql.TxOptions and error will both be nil.
func (cmd *Command) ValidateSQL(dialect string) (*sql.TxOptions, error) {
err := cmd.Validate()
if err != nil {
return nil, err
}
mode, err := sqlMode(cmd.Mode)
if err != nil {
return nil, err
}
// Mutates Arguments to only use positional arguments with the type sql.NamedArg.
// This is a required by the sql.Driver before submitting arguments.
cmd.Arguments.sql(dialect)
iso, err := sqlIsolation(cmd.Isolation)
if err != nil {
return nil, err
}
// When isolation is out-of-range, this is indicative that no
// transaction should be executed and sql.TxOptions should be nil.
if iso < sql.LevelDefault {
return nil, nil
}
// In query mode, execute the transaction in read-only, unless it's Microsoft SQL
// which does not support that type of transaction.
readOnly := mode == "query" && dialect != "mssql"
return &sql.TxOptions{Isolation: iso, ReadOnly: readOnly}, nil
}
// sqlConn gets the next available sql.Conn in the connection pool and runs a function to use it.
//
// If the transaction options are nil, run the useIt function outside a transaction.
// This is potentially an unsafe operation if the command does not clean up its state.
func sqlConn(ctx context.Context, driver *sqlx.DB, txx *sql.TxOptions, useIt func(*sql.Conn) error) error {
conn, err := driver.Conn(ctx)
if err != nil {
return err
}
defer conn.Close()
// If transaction options are specified, begin and defer a rollback to catch errors.
var tx *sql.Tx
if txx != nil {
tx, err = conn.BeginTx(ctx, txx)
if err != nil {
return err
}
defer tx.Rollback()
}
err = useIt(conn)
// Check if useIt was successful and a transaction exists before committing.
if err == nil && tx != nil {
err = tx.Commit()
}
return err
}
// sqlQuery queries rows on a sql.Conn and returns an array of result objects.
func sqlQuery(ctx context.Context, conn *sql.Conn, stmt string, args []interface{}) ([]map[string]interface{}, error) {
rows, err := conn.QueryContext(ctx, stmt, args...)
if err == nil {
return sqlRows(rows)
}
return nil, err
}
// sqlExec executes a command on a sql.Conn and returns the result of the operation.
func sqlExec(ctx context.Context, conn *sql.Conn, stmt string, args []interface{}) (sqlResult, error) {
exec, err := conn.ExecContext(ctx, stmt, args...)
if err == nil {
return sqlResultFrom(exec), nil
}
return sqlResult{}, err
}
// sql mutates Arguments to contain a positional []sql.NamedArg.
//
// The actual return type is []interface{} due to the native Golang
// function signatures for sql.Exec and sql.Query being generic.
func (args *Arguments) sql(dialect string) {
result := args.Positional
for i, val := range result {
result[i] = sqlArg("", val, dialect)
}
for key, val := range args.Named {
result = append(result, sqlArg(key, val, dialect))
}
args.Positional = result
args.Named = map[string]interface{}{}
}
// sqlArg creates a sql.NamedArg from a key-value pair and an optional dialect.
//
// Certain dialects will need to wrap objects, such as arrays, to conform its driver requirements.
func sqlArg(key, val interface{}, dialect string) sql.NamedArg {
switch reflect.ValueOf(val).Kind() {
// PostgreSQL and Clickhouse require arrays to be wrapped before
// being inserted into the driver interface.
case reflect.Slice, reflect.Array:
switch dialect {
case "postgres":
val = pq.Array(val)
case "clickhouse":
val = clickhouse.Array(val)
}
}
return sql.Named(fmt.Sprint(key), val)
}
// sqlIsolation tries to match a string to a sql.IsolationLevel.
func sqlIsolation(str string) (sql.IsolationLevel, error) {
if str == "none" {
return sql.IsolationLevel(-1), nil
}
for iso := sql.LevelDefault; ; iso++ {
if iso > sql.LevelLinearizable {
return -1, fmt.Errorf("cannot provide an invalid sql isolation level: '%s'", str)
}
if str == "" || strings.EqualFold(iso.String(), strings.ReplaceAll(str, "_", " ")) {
return iso, nil
}
}
}
// sqlMode tries to match a string to a command mode: 'query' or 'exec' for now.
func sqlMode(str string) (string, error) {
switch str {
case "query", "exec":
return str, nil
default:
return "", fmt.Errorf("cannot provide invalid sql mode: '%s'", str)
}
}
// sqlRows scans through a SQL result set and returns an array of objects.
func sqlRows(rows *sql.Rows) ([]map[string]interface{}, error) {
columns, err := rows.Columns()
if err != nil {
return nil, errors.Wrap(err, "could not extract columns from result")
}
defer rows.Close()
types, err := rows.ColumnTypes()
if err != nil {
// Some drivers do not support type extraction, so fail silently and continue.
types = make([]*sql.ColumnType, len(columns))
}
values := make([]interface{}, len(columns))
pointers := make([]interface{}, len(columns))
var results []map[string]interface{}
for rows.Next() {
for i := range columns {
pointers[i] = &values[i]
}
rows.Scan(pointers...)
// Convert a row, an array of values, into an object where
// each key is the name of its respective column.
entry := make(map[string]interface{})
for i, col := range columns {
entry[col] = sqlValue(values[i], types[i])
}
results = append(results, entry)
}
return results, nil
}
// sqlValue handles special cases where sql.Rows does not return a "human-readable" object.
func sqlValue(val interface{}, col *sql.ColumnType) interface{} {
bytes, ok := val.([]byte)
if ok {
// Opportunistically check for embeded JSON and convert it to a first-class object.
var embeded interface{}
if json.Unmarshal(bytes, &embeded) == nil {
return embeded
}
// STOR-604: investigate a way to coerce PostgreSQL arrays '{a, b, ...}' into JSON.
// Although easy with strings, it becomes more difficult with special types like INET[].
return string(bytes)
}
return val
}
// sqlResult is a thin wrapper around sql.Result.
type sqlResult struct {
LastInsertId int64 `json:"last_insert_id"`
RowsAffected int64 `json:"rows_affected"`
}
// sqlResultFrom converts sql.Result into a JSON-marshable sqlResult.
func sqlResultFrom(res sql.Result) sqlResult {
insertID, errID := res.LastInsertId()
rowsAffected, errRows := res.RowsAffected()
// If an error occurs when extracting the result, it is because the
// driver does not support that specific field. Instead of passing this
// to the user, omit the field in the response.
if errID != nil {
insertID = -1
}
if errRows != nil {
rowsAffected = -1
}
return sqlResult{insertID, rowsAffected}
}
+336
View File
@@ -0,0 +1,336 @@
package dbconnect
import (
"context"
"database/sql"
"fmt"
"net/url"
"strings"
"testing"
"time"
"github.com/kshvakov/clickhouse"
"github.com/lib/pq"
"github.com/DATA-DOG/go-sqlmock"
"github.com/stretchr/testify/assert"
)
func TestNewSQLClient(t *testing.T) {
originURLs := []string{
"postgres://localhost",
"cockroachdb://localhost:1337",
"postgresql://user:pass@127.0.0.1",
"mysql://localhost",
"clickhouse://127.0.0.1:9000/?debug",
"sqlite3::memory:",
"file:test.db?cache=shared",
}
for _, originURL := range originURLs {
origin, _ := url.Parse(originURL)
_, err := NewSQLClient(context.Background(), origin)
assert.NoError(t, err, originURL)
}
originURLs = []string{
"",
"/",
"http://localhost",
"coolthing://user:pass@127.0.0.1",
}
for _, originURL := range originURLs {
origin, _ := url.Parse(originURL)
_, err := NewSQLClient(context.Background(), origin)
assert.Error(t, err, originURL)
}
}
func TestArgumentsSQL(t *testing.T) {
args := []Arguments{
Arguments{
Positional: []interface{}{
"val", 10, 3.14,
},
},
Arguments{
Named: map[string]interface{}{
"key": time.Unix(0, 0),
},
},
}
var nameType sql.NamedArg
for _, arg := range args {
arg.sql("")
for _, named := range arg.Positional {
assert.IsType(t, nameType, named)
}
}
}
func TestSQLArg(t *testing.T) {
tests := []struct {
key interface{}
val interface{}
dialect string
arg sql.NamedArg
}{
{"key", "val", "mssql", sql.Named("key", "val")},
{0, 1, "sqlite3", sql.Named("0", 1)},
{1, []string{"a", "b", "c"}, "postgres", sql.Named("1", pq.Array([]string{"a", "b", "c"}))},
{"in", []uint{0, 1}, "clickhouse", sql.Named("in", clickhouse.Array([]uint{0, 1}))},
{"", time.Unix(0, 0), "", sql.Named("", time.Unix(0, 0))},
}
for _, test := range tests {
arg := sqlArg(test.key, test.val, test.dialect)
assert.Equal(t, test.arg, arg, test.key)
}
}
func TestSQLIsolation(t *testing.T) {
tests := []struct {
str string
iso sql.IsolationLevel
}{
{"", sql.LevelDefault},
{"DEFAULT", sql.LevelDefault},
{"read_UNcommitted", sql.LevelReadUncommitted},
{"serializable", sql.LevelSerializable},
{"none", sql.IsolationLevel(-1)},
{"SNAP shot", -2},
{"blah", -2},
}
for _, test := range tests {
iso, err := sqlIsolation(test.str)
if test.iso < -1 {
assert.Error(t, err, test.str)
} else {
assert.NoError(t, err)
assert.Equal(t, test.iso, iso, test.str)
}
}
}
func TestSQLMode(t *testing.T) {
modes := []string{
"query",
"exec",
}
for _, mode := range modes {
actual, err := sqlMode(mode)
assert.NoError(t, err)
assert.Equal(t, strings.ToLower(mode), actual, mode)
}
modes = []string{
"",
"blah",
}
for _, mode := range modes {
_, err := sqlMode(mode)
assert.Error(t, err)
}
}
func helperRows(mockRows *sqlmock.Rows) *sql.Rows {
db, mock, _ := sqlmock.New()
mock.ExpectQuery("SELECT").WillReturnRows(mockRows)
rows, _ := db.Query("SELECT")
return rows
}
func TestSQLRows(t *testing.T) {
actual, err := sqlRows(helperRows(sqlmock.NewRows(
[]string{"name", "age", "dept"}).
AddRow("alice", 19, "prod")))
expected := []map[string]interface{}{map[string]interface{}{
"name": "alice",
"age": int64(19),
"dept": "prod"}}
assert.NoError(t, err)
assert.ElementsMatch(t, expected, actual)
}
func TestSQLValue(t *testing.T) {
tests := []struct {
input interface{}
output interface{}
}{
{"hello", "hello"},
{1, 1},
{false, false},
{[]byte("random"), "random"},
{[]byte("{\"json\":true}"), map[string]interface{}{"json": true}},
{[]byte("[]"), []interface{}{}},
}
for _, test := range tests {
assert.Equal(t, test.output, sqlValue(test.input, nil), test.input)
}
}
func TestSQLResultFrom(t *testing.T) {
res := sqlResultFrom(sqlmock.NewResult(1, 2))
assert.Equal(t, sqlResult{1, 2}, res)
res = sqlResultFrom(sqlmock.NewErrorResult(fmt.Errorf("")))
assert.Equal(t, sqlResult{-1, -1}, res)
}
func helperSQLite3(t *testing.T) (context.Context, Client) {
t.Helper()
ctx := context.Background()
url, _ := url.Parse("file::memory:?cache=shared")
sqlite3, err := NewSQLClient(ctx, url)
assert.NoError(t, err)
return ctx, sqlite3
}
func TestPing(t *testing.T) {
ctx, sqlite3 := helperSQLite3(t)
err := sqlite3.Ping(ctx)
assert.NoError(t, err)
}
func TestSubmit(t *testing.T) {
ctx, sqlite3 := helperSQLite3(t)
res, err := sqlite3.Submit(ctx, &Command{
Statement: "CREATE TABLE t (a INTEGER, b FLOAT, c TEXT, d BLOB);",
Mode: "exec",
})
assert.NoError(t, err)
assert.Equal(t, sqlResult{0, 0}, res)
res, err = sqlite3.Submit(ctx, &Command{
Statement: "SELECT * FROM t;",
Mode: "query",
})
assert.NoError(t, err)
assert.Empty(t, res)
res, err = sqlite3.Submit(ctx, &Command{
Statement: "INSERT INTO t VALUES (?, ?, ?, ?);",
Mode: "exec",
Arguments: Arguments{
Positional: []interface{}{
1,
3.14,
"text",
"blob",
},
},
})
assert.NoError(t, err)
assert.Equal(t, sqlResult{1, 1}, res)
res, err = sqlite3.Submit(ctx, &Command{
Statement: "UPDATE t SET c = NULL;",
Mode: "exec",
})
assert.NoError(t, err)
assert.Equal(t, sqlResult{1, 1}, res)
res, err = sqlite3.Submit(ctx, &Command{
Statement: "SELECT * FROM t WHERE a = ?;",
Mode: "query",
Arguments: Arguments{
Positional: []interface{}{1},
},
})
assert.NoError(t, err)
assert.Len(t, res, 1)
resf, ok := res.([]map[string]interface{})
assert.True(t, ok)
assert.EqualValues(t, map[string]interface{}{
"a": int64(1),
"b": 3.14,
"c": nil,
"d": "blob",
}, resf[0])
res, err = sqlite3.Submit(ctx, &Command{
Statement: "DROP TABLE t;",
Mode: "exec",
})
assert.NoError(t, err)
assert.Equal(t, sqlResult{1, 1}, res)
}
func TestSubmitTransaction(t *testing.T) {
ctx, sqlite3 := helperSQLite3(t)
res, err := sqlite3.Submit(ctx, &Command{
Statement: "BEGIN;",
Mode: "exec",
})
assert.Error(t, err)
assert.Empty(t, res)
res, err = sqlite3.Submit(ctx, &Command{
Statement: "BEGIN; CREATE TABLE tt (a INT); COMMIT;",
Mode: "exec",
Isolation: "none",
})
assert.NoError(t, err)
assert.Equal(t, sqlResult{0, 0}, res)
rows, err := sqlite3.Submit(ctx, &Command{
Statement: "SELECT * FROM tt;",
Mode: "query",
Isolation: "repeatable_read",
})
assert.NoError(t, err)
assert.Empty(t, rows)
}
func TestSubmitTimeout(t *testing.T) {
ctx, sqlite3 := helperSQLite3(t)
res, err := sqlite3.Submit(ctx, &Command{
Statement: "SELECT * FROM t;",
Mode: "query",
Timeout: 1 * time.Nanosecond,
})
assert.Error(t, err)
assert.Empty(t, res)
}
func TestSubmitMode(t *testing.T) {
ctx, sqlite3 := helperSQLite3(t)
res, err := sqlite3.Submit(ctx, &Command{
Statement: "SELECT * FROM t;",
Mode: "notanoption",
})
assert.Error(t, err)
assert.Empty(t, res)
}
func TestSubmitEmpty(t *testing.T) {
ctx, sqlite3 := helperSQLite3(t)
res, err := sqlite3.Submit(ctx, &Command{
Statement: "; ; ; ;",
Mode: "query",
})
assert.Error(t, err)
assert.Empty(t, res)
}
+78
View File
@@ -0,0 +1,78 @@
# docker-compose -f ./dbconnect_tests/dbconnect.yaml up --build --force-recreate --renew-anon-volumes --exit-code-from cloudflared
version: "2.3"
networks:
test-dbconnect-network:
driver: bridge
services:
cloudflared:
build:
context: ../
dockerfile: dev.Dockerfile
command: go test github.com/cloudflare/cloudflared/dbconnect_tests -v
depends_on:
postgres:
condition: service_healthy
mysql:
condition: service_healthy
mssql:
condition: service_healthy
clickhouse:
condition: service_healthy
environment:
DBCONNECT_INTEGRATION_TEST: "true"
POSTGRESQL_URL: postgres://postgres:secret@postgres/db?sslmode=disable
MYSQL_URL: mysql://root:secret@mysql/db?tls=false
MSSQL_URL: mssql://sa:secret12345!@mssql
CLICKHOUSE_URL: clickhouse://clickhouse:9000/db
networks:
- test-dbconnect-network
postgres:
image: postgres:11.4-alpine
environment:
POSTGRES_DB: db
POSTGRES_PASSWORD: secret
healthcheck:
test: ["CMD", "pg_isready", "-U", "postgres"]
start_period: 3s
interval: 1s
timeout: 3s
retries: 10
networks:
- test-dbconnect-network
mysql:
image: mysql:8.0
environment:
MYSQL_DATABASE: db
MYSQL_ROOT_PASSWORD: secret
healthcheck:
test: ["CMD", "mysqladmin", "ping"]
start_period: 3s
interval: 1s
timeout: 3s
retries: 10
networks:
- test-dbconnect-network
mssql:
image: mcr.microsoft.com/mssql/server:2017-CU8-ubuntu
environment:
ACCEPT_EULA: "Y"
SA_PASSWORD: secret12345!
healthcheck:
test: ["CMD", "/opt/mssql-tools/bin/sqlcmd", "-S", "localhost", "-U", "sa", "-P", "secret12345!", "-Q", "SELECT 1"]
start_period: 3s
interval: 1s
timeout: 3s
retries: 10
networks:
- test-dbconnect-network
clickhouse:
image: yandex/clickhouse-server:19.11
healthcheck:
test: ["CMD", "clickhouse-client", "--query", "SELECT 1"]
start_period: 3s
interval: 1s
timeout: 3s
retries: 10
networks:
- test-dbconnect-network
+265
View File
@@ -0,0 +1,265 @@
package dbconnect_tests
import (
"context"
"log"
"net/url"
"os"
"testing"
"github.com/stretchr/testify/assert"
"github.com/cloudflare/cloudflared/dbconnect"
)
func TestIntegrationPostgreSQL(t *testing.T) {
ctx, pq := helperNewSQLClient(t, "POSTGRESQL_URL")
err := pq.Ping(ctx)
assert.NoError(t, err)
_, err = pq.Submit(ctx, &dbconnect.Command{
Statement: "CREATE TABLE t (a TEXT, b UUID, c JSON, d INET[], e SERIAL);",
Mode: "exec",
})
assert.NoError(t, err)
_, err = pq.Submit(ctx, &dbconnect.Command{
Statement: "INSERT INTO t VALUES ($1, $2, $3, $4);",
Mode: "exec",
Arguments: dbconnect.Arguments{
Positional: []interface{}{
"text",
"6b8d686d-bd8e-43bc-b09a-cfcbbe702c10",
"{\"bool\":true,\"array\":[\"a\", 1, 3.14],\"embed\":{\"num\":21}}",
[]string{"1.1.1.1", "1.0.0.1"},
},
},
})
assert.NoError(t, err)
_, err = pq.Submit(ctx, &dbconnect.Command{
Statement: "UPDATE t SET b = NULL;",
Mode: "exec",
})
assert.NoError(t, err)
res, err := pq.Submit(ctx, &dbconnect.Command{
Statement: "SELECT * FROM t;",
Mode: "query",
})
assert.NoError(t, err)
assert.IsType(t, make([]map[string]interface{}, 0), res)
actual := res.([]map[string]interface{})[0]
expected := map[string]interface{}{
"a": "text",
"b": nil,
"c": map[string]interface{}{
"bool": true,
"array": []interface{}{"a", float64(1), 3.14},
"embed": map[string]interface{}{"num": float64(21)},
},
"d": "{1.1.1.1,1.0.0.1}",
"e": int64(1),
}
assert.EqualValues(t, expected, actual)
_, err = pq.Submit(ctx, &dbconnect.Command{
Statement: "DROP TABLE t;",
Mode: "exec",
})
assert.NoError(t, err)
}
func TestIntegrationMySQL(t *testing.T) {
ctx, my := helperNewSQLClient(t, "MYSQL_URL")
err := my.Ping(ctx)
assert.NoError(t, err)
_, err = my.Submit(ctx, &dbconnect.Command{
Statement: "CREATE TABLE t (a CHAR, b TINYINT, c FLOAT, d JSON, e YEAR);",
Mode: "exec",
})
assert.NoError(t, err)
_, err = my.Submit(ctx, &dbconnect.Command{
Statement: "INSERT INTO t VALUES (?, ?, ?, ?, ?);",
Mode: "exec",
Arguments: dbconnect.Arguments{
Positional: []interface{}{
"a",
10,
3.14,
"{\"bool\":true}",
2000,
},
},
})
assert.NoError(t, err)
_, err = my.Submit(ctx, &dbconnect.Command{
Statement: "ALTER TABLE t ADD COLUMN f GEOMETRY;",
Mode: "exec",
})
assert.NoError(t, err)
res, err := my.Submit(ctx, &dbconnect.Command{
Statement: "SELECT * FROM t;",
Mode: "query",
})
assert.NoError(t, err)
assert.IsType(t, make([]map[string]interface{}, 0), res)
actual := res.([]map[string]interface{})[0]
expected := map[string]interface{}{
"a": "a",
"b": float64(10),
"c": 3.14,
"d": map[string]interface{}{"bool": true},
"e": float64(2000),
"f": nil,
}
assert.EqualValues(t, expected, actual)
_, err = my.Submit(ctx, &dbconnect.Command{
Statement: "DROP TABLE t;",
Mode: "exec",
})
assert.NoError(t, err)
}
func TestIntegrationMSSQL(t *testing.T) {
ctx, ms := helperNewSQLClient(t, "MSSQL_URL")
err := ms.Ping(ctx)
assert.NoError(t, err)
_, err = ms.Submit(ctx, &dbconnect.Command{
Statement: "CREATE TABLE t (a BIT, b DECIMAL, c MONEY, d TEXT);",
Mode: "exec"})
assert.NoError(t, err)
_, err = ms.Submit(ctx, &dbconnect.Command{
Statement: "INSERT INTO t VALUES (?, ?, ?, ?);",
Mode: "exec",
Arguments: dbconnect.Arguments{
Positional: []interface{}{
0,
3,
"$0.99",
"text",
},
},
})
assert.NoError(t, err)
_, err = ms.Submit(ctx, &dbconnect.Command{
Statement: "UPDATE t SET d = NULL;",
Mode: "exec",
})
assert.NoError(t, err)
res, err := ms.Submit(ctx, &dbconnect.Command{
Statement: "SELECT * FROM t;",
Mode: "query",
})
assert.NoError(t, err)
assert.IsType(t, make([]map[string]interface{}, 0), res)
actual := res.([]map[string]interface{})[0]
expected := map[string]interface{}{
"a": false,
"b": float64(3),
"c": float64(0.99),
"d": nil,
}
assert.EqualValues(t, expected, actual)
_, err = ms.Submit(ctx, &dbconnect.Command{
Statement: "DROP TABLE t;",
Mode: "exec",
})
assert.NoError(t, err)
}
func TestIntegrationClickhouse(t *testing.T) {
ctx, ch := helperNewSQLClient(t, "CLICKHOUSE_URL")
err := ch.Ping(ctx)
assert.NoError(t, err)
_, err = ch.Submit(ctx, &dbconnect.Command{
Statement: "CREATE TABLE t (a UUID, b String, c Float64, d UInt32, e Int16, f Array(Enum8('a'=1, 'b'=2, 'c'=3))) engine=Memory;",
Mode: "exec",
})
assert.NoError(t, err)
_, err = ch.Submit(ctx, &dbconnect.Command{
Statement: "INSERT INTO t VALUES (?, ?, ?, ?, ?, ?);",
Mode: "exec",
Arguments: dbconnect.Arguments{
Positional: []interface{}{
"ec65f626-6f50-4c86-9628-6314ef1edacd",
"",
3.14,
314,
-144,
[]string{"a", "b", "c"},
},
},
})
assert.NoError(t, err)
res, err := ch.Submit(ctx, &dbconnect.Command{
Statement: "SELECT * FROM t;",
Mode: "query",
})
assert.NoError(t, err)
assert.IsType(t, make([]map[string]interface{}, 0), res)
actual := res.([]map[string]interface{})[0]
expected := map[string]interface{}{
"a": "ec65f626-6f50-4c86-9628-6314ef1edacd",
"b": "",
"c": float64(3.14),
"d": uint32(314),
"e": int16(-144),
"f": []string{"a", "b", "c"},
}
assert.EqualValues(t, expected, actual)
_, err = ch.Submit(ctx, &dbconnect.Command{
Statement: "DROP TABLE t;",
Mode: "exec",
})
assert.NoError(t, err)
}
func helperNewSQLClient(t *testing.T, env string) (context.Context, dbconnect.Client) {
_, ok := os.LookupEnv("DBCONNECT_INTEGRATION_TEST")
if ok {
t.Helper()
} else {
t.SkipNow()
}
val, ok := os.LookupEnv(env)
if !ok {
log.Fatalf("must provide database url as environment variable: %s", env)
}
parsed, err := url.Parse(val)
if err != nil {
log.Fatalf("cannot provide invalid database url: %s=%s", env, val)
}
ctx := context.Background()
client, err := dbconnect.NewSQLClient(ctx, parsed)
if err != nil {
log.Fatalf("could not start test client: %s", err)
}
return ctx, client
}
+4
View File
@@ -0,0 +1,4 @@
FROM golang:1.12 as builder
WORKDIR /go/src/github.com/cloudflare/cloudflared/
RUN apt-get update
COPY . .
+72
View File
@@ -0,0 +1,72 @@
module github.com/cloudflare/cloudflared
go 1.12
require (
github.com/DATA-DOG/go-sqlmock v1.3.3
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239 // indirect
github.com/aws/aws-sdk-go v1.25.8
github.com/beorn7/perks v1.0.1 // indirect
github.com/certifi/gocertifi v0.0.0-20180118203423-deb3ae2ef261 // indirect
github.com/cloudflare/brotli-go v0.0.0-20191101163834-d34379f7ff93
github.com/cloudflare/golibs v0.0.0-20170913112048-333127dbecfc
github.com/coredns/coredns v1.2.0
github.com/coreos/go-oidc v0.0.0-20171002155002-a93f71fdfe73
github.com/coreos/go-systemd v0.0.0-20190620071333-e64a0ec8b42a
github.com/denisenkom/go-mssqldb v0.0.0-20191001013358-cfbb681360f0
github.com/equinox-io/equinox v1.2.0
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51 // indirect
github.com/facebookgo/freeport v0.0.0-20150612182905-d4adf43b75b9 // indirect
github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 // indirect
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870 // indirect
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568 // indirect
github.com/getsentry/raven-go v0.0.0-20180517221441-ed7bcb39ff10
github.com/gliderlabs/ssh v0.0.0-20191009160644-63518b5243e0
github.com/go-sql-driver/mysql v1.4.1
github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
github.com/google/uuid v1.1.1
github.com/gorilla/mux v1.7.3
github.com/gorilla/websocket v1.2.0
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645 // indirect
github.com/jmoiron/sqlx v1.2.0
github.com/konsorten/go-windows-terminal-sequences v1.0.2 // indirect
github.com/kshvakov/clickhouse v1.3.11
github.com/lib/pq v1.2.0
github.com/mattn/go-colorable v0.1.4
github.com/mattn/go-isatty v0.0.10 // indirect
github.com/mattn/go-sqlite3 v1.11.0
github.com/mholt/caddy v0.0.0-20180807230124-d3b731e9255b // indirect
github.com/miekg/dns v1.1.8
github.com/mitchellh/go-homedir v1.1.0
github.com/opentracing/opentracing-go v1.1.0 // indirect
github.com/philhofer/fwd v1.0.0 // indirect
github.com/pkg/errors v0.8.1
github.com/pquerna/cachecontrol v0.0.0-20180517163645-1555304b9b35 // indirect
github.com/prometheus/client_golang v1.0.0
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4 // indirect
github.com/prometheus/common v0.7.0 // indirect
github.com/prometheus/procfs v0.0.5 // indirect
github.com/rifflock/lfshook v0.0.0-20180920164130-b9218ef580f5
github.com/sirupsen/logrus v1.4.2
github.com/stretchr/testify v1.3.0
github.com/tinylib/msgp v1.1.0 // indirect
github.com/xo/dburl v0.0.0-20191005012637-293c3298d6c0
golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc
golang.org/x/net v0.0.0-20191007182048-72f939374954
golang.org/x/sync v0.0.0-20190423024810-112230192c58
golang.org/x/sys v0.0.0-20191008105621-543471e840be
golang.org/x/text v0.3.2 // indirect
google.golang.org/genproto v0.0.0-20191007204434-a023cd5227bd // indirect
google.golang.org/grpc v1.24.0 // indirect
gopkg.in/coreos/go-oidc.v2 v2.1.0
gopkg.in/square/go-jose.v2 v2.4.0 // indirect
gopkg.in/urfave/cli.v2 v2.0.0-20180128181224-d604b6ffeee8
gopkg.in/yaml.v2 v2.2.4 // indirect
zombiezen.com/go/capnproto2 v0.0.0-20180616160808-7cfd211c19c7
)
// ../../go/pkg/mod/github.com/coredns/coredns@v1.2.0/plugin/metrics/metrics.go:40:49: too many arguments in call to prometheus.NewProcessCollector
// have (int, string)
// want (prometheus.ProcessCollectorOpts)
replace github.com/prometheus/client_golang => github.com/prometheus/client_golang v0.9.0-pre1
+210
View File
@@ -0,0 +1,210 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/DATA-DOG/go-sqlmock v1.3.3 h1:CWUqKXe0s8A2z6qCgkP4Kru7wC11YoAnoupUKFDnH08=
github.com/DATA-DOG/go-sqlmock v1.3.3/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239 h1:kFOfPq6dUM1hTo4JG6LR5AXSUEsOjtdm0kw0FtQtMJA=
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c=
github.com/aws/aws-sdk-go v1.25.8 h1:n7I+HUUXjun2CsX7JK+1hpRIkZrlKhd3nayeb+Xmavs=
github.com/aws/aws-sdk-go v1.25.8/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bkaradzic/go-lz4 v1.0.0 h1:RXc4wYsyz985CkXXeX04y4VnZFGG8Rd43pRaHsOXAKk=
github.com/bkaradzic/go-lz4 v1.0.0/go.mod h1:0YdlkowM3VswSROI7qDxhRvJ3sLhlFrRRwjwegp5jy4=
github.com/certifi/gocertifi v0.0.0-20180118203423-deb3ae2ef261 h1:6/yVvBsKeAw05IUj4AzvrxaCnDjN4nUqKjW9+w5wixg=
github.com/certifi/gocertifi v0.0.0-20180118203423-deb3ae2ef261/go.mod h1:GJKEexRPVJrBSOjoqN5VNOIKJ5Q3RViH6eu3puDRwx4=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudflare/brotli-go v0.0.0-20191101163834-d34379f7ff93 h1:QrGfkZDnMxcWHaYDdB7CmqS9i26OAnUj/xcus/abYkY=
github.com/cloudflare/brotli-go v0.0.0-20191101163834-d34379f7ff93/go.mod h1:QiTe66jFdP7cUKMCCf/WrvDyYdtdmdZfVcdoLbzaKVY=
github.com/cloudflare/golibs v0.0.0-20170913112048-333127dbecfc h1:Dvk3ySBsOm5EviLx6VCyILnafPcQinXGP5jbTdHUJgE=
github.com/cloudflare/golibs v0.0.0-20170913112048-333127dbecfc/go.mod h1:HlgKKR8V5a1wroIDDIz3/A+T+9Janfq+7n1P5sEFdi0=
github.com/cloudflare/golz4 v0.0.0-20150217214814-ef862a3cdc58 h1:F1EaeKL/ta07PY/k9Os/UFtwERei2/XzGemhpGnBKNg=
github.com/cloudflare/golz4 v0.0.0-20150217214814-ef862a3cdc58/go.mod h1:EOBUe0h4xcZ5GoxqC5SDxFQ8gwyZPKQoEzownBlhI80=
github.com/coredns/coredns v1.2.0 h1:YEI38K2BJYzL/SxO2tZFD727T/C68DqVWkBQjT0sWPU=
github.com/coredns/coredns v1.2.0/go.mod h1:zASH/MVDgR6XZTbxvOnsZfffS+31vg6Ackf/wo1+AM0=
github.com/coreos/go-oidc v0.0.0-20171002155002-a93f71fdfe73 h1:7CNPV0LWRCa1FNmqg700pbXhzvmoaXKyfxWRkjRym7Q=
github.com/coreos/go-oidc v0.0.0-20171002155002-a93f71fdfe73/go.mod h1:CgnwVTmzoESiwO9qyAFEMiHoZ1nMCKZlZ9V6mm3/LKc=
github.com/coreos/go-systemd v0.0.0-20190620071333-e64a0ec8b42a h1:W8b4lQ4tFF21aspRGoBuCNV6V2fFJBF+pm1J6OY8Lys=
github.com/coreos/go-systemd v0.0.0-20190620071333-e64a0ec8b42a/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/denisenkom/go-mssqldb v0.0.0-20191001013358-cfbb681360f0 h1:epsH3lb7KVbXHYk7LYGN5EiE0MxcevHU85CKITJ0wUY=
github.com/denisenkom/go-mssqldb v0.0.0-20191001013358-cfbb681360f0/go.mod h1:xbL0rPBG9cCiLr28tMa8zpbdarY27NDyej4t/EjAShU=
github.com/equinox-io/equinox v1.2.0 h1:bBS7Ou+Y7Jwgmy8TWSYxEh85WctuFn7FPlgbUzX4DBA=
github.com/equinox-io/equinox v1.2.0/go.mod h1:6s3HJB0PYUNgs0mxmI8fHdfVl3TQ25ieA/PVfr+eyVo=
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51 h1:0JZ+dUmQeA8IIVUMzysrX4/AKuQwWhV2dYQuPZdvdSQ=
github.com/facebookgo/ensure v0.0.0-20160127193407-b4ab57deab51/go.mod h1:Yg+htXGokKKdzcwhuNDwVvN+uBxDGXJ7G/VN1d8fa64=
github.com/facebookgo/freeport v0.0.0-20150612182905-d4adf43b75b9 h1:wWke/RUCl7VRjQhwPlR/v0glZXNYzBHdNUzf/Am2Nmg=
github.com/facebookgo/freeport v0.0.0-20150612182905-d4adf43b75b9/go.mod h1:uPmAp6Sws4L7+Q/OokbWDAK1ibXYhB3PXFP1kol5hPg=
github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434 h1:mOp33BLbcbJ8fvTAmZacbBiOASfxN+MLcLxymZCIrGE=
github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434/go.mod h1:KigFdumBXUPSwzLDbeuzyt0elrL7+CP7TKuhrhT4bcU=
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052 h1:JWuenKqqX8nojtoVVWjGfOF9635RETekkoH6Cc9SX0A=
github.com/facebookgo/stack v0.0.0-20160209184415-751773369052/go.mod h1:UbMTZqLaRiH3MsBH8va0n7s1pQYcu3uTb8G4tygF4Zg=
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870 h1:E2s37DuLxFhQDg5gKsWoLBOB0n+ZW8s599zru8FJ2/Y=
github.com/facebookgo/subset v0.0.0-20150612182917-8dac2c3c4870/go.mod h1:5tD+neXqOorC30/tWg0LCSkrqj/AR6gu8yY8/fpw1q0=
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568 h1:BHsljHzVlRcyQhjrss6TZTdY2VfCqZPbv5k3iBFa2ZQ=
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568/go.mod h1:xEzjJPgXI435gkrCt3MPfRiAkVrwSbHsst4LCFVfpJc=
github.com/getsentry/raven-go v0.0.0-20180517221441-ed7bcb39ff10 h1:YO10pIIBftO/kkTFdWhctH96grJ7qiy7bMdiZcIvPKs=
github.com/getsentry/raven-go v0.0.0-20180517221441-ed7bcb39ff10/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49PX4NzFV5kcQ=
github.com/gliderlabs/ssh v0.0.0-20191009160644-63518b5243e0 h1:gF8ngtda767ddth2SH0YSAhswhz6qUkvyI9EZFYCWJA=
github.com/gliderlabs/ssh v0.0.0-20191009160644-63518b5243e0/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.4.1 h1:g24URVg0OFbNUTx9qqY1IRZ9D9z3iPyi5zKhQZpNwpA=
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3 h1:zN2lZNZRflqFyxVaTIU61KNKQ9C0055u9CAfpmqUvo4=
github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3/go.mod h1:nPpo7qLxd6XL3hWJG/O60sR8ZKfMCiIoNap5GvD12KU=
github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe h1:lXe2qZdvpiX5WZkZR4hgp4KJVfY3nMkvmwbVkpv1rVY=
github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/mux v1.7.3 h1:gnP5JzjVOuiZD07fKKToCAOjS0yOpj/qPETTXCCS6hw=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/websocket v1.2.0 h1:VJtLvh6VQym50czpZzx07z/kw9EgAxI3x1ZB8taTMQQ=
github.com/gorilla/websocket v1.2.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645 h1:MJG/KsmcqMwFAkh8mTnAwhyKoB+sTAnY4CACC110tbU=
github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645/go.mod h1:6iZfnjpejD4L/4DwD7NryNaJyCQdzwWwH2MWhCA90Kw=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5imkbgOkpRUYLnmbU7UEFbjtDA2hxJ1ichM=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jmoiron/sqlx v1.2.0 h1:41Ip0zITnmWNR/vHV+S4m+VoUivnWY5E4OJfLZjCJMA=
github.com/jmoiron/sqlx v1.2.0/go.mod h1:1FEQNm3xlJgrMD+FBdI9+xvCksHtbpVBBw5dYhBSsks=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.2 h1:DB17ag19krx9CFsz4o3enTrPXyIXCl+2iCXH/aMAp9s=
github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kshvakov/clickhouse v1.3.11 h1:dtzTJY0fCA+MWkLyuKZaNPkmSwdX4gh8+Klic9NB1Lw=
github.com/kshvakov/clickhouse v1.3.11/go.mod h1:/SVBAcqF3u7rxQ9sTWCZwf8jzzvxiZGeQvtmSF2BBEc=
github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348 h1:MtvEpTB6LX3vkb4ax0b5D2DHbNAUsen0Gx5wZoq3lV4=
github.com/kylelemons/godebug v0.0.0-20170820004349-d65d576e9348/go.mod h1:B69LEHPfb2qLo0BaaOLcbitczOKLWTsrBG9LczfCD4k=
github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/lib/pq v1.2.0 h1:LXpIM/LZ5xGFhOpXAQUIMM1HdyqzVYM13zNdjCEEcA0=
github.com/lib/pq v1.2.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.10 h1:qxFzApOv4WsAL965uUPIsXzAKCZxN2p9UqdhFS4ZW10=
github.com/mattn/go-isatty v0.0.10/go.mod h1:qgIWMr58cqv1PHHyhnkY9lrL7etaEgOFcMEpPG5Rm84=
github.com/mattn/go-sqlite3 v1.9.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
github.com/mattn/go-sqlite3 v1.11.0 h1:LDdKkqtYlom37fkvqs8rMPFKAMe8+SgjbwZ6ex1/A/Q=
github.com/mattn/go-sqlite3 v1.11.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mholt/caddy v0.0.0-20180807230124-d3b731e9255b h1:/BbY4n99iMazlr2igipph+hj0MwlZIWpcsP8Iy+na+s=
github.com/mholt/caddy v0.0.0-20180807230124-d3b731e9255b/go.mod h1:Wb1PlT4DAYSqOEd03MsqkdkXnTxA8v9pKjdpxbqM1kY=
github.com/miekg/dns v1.1.8 h1:1QYRAKU3lN5cRfLCkPU08hwvLJFhvjP6MqNMmQz6ZVI=
github.com/miekg/dns v1.1.8/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/opentracing/opentracing-go v1.1.0 h1:pWlfV3Bxv7k65HYwkikxat0+s3pV4bsqf19k25Ur8rU=
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/philhofer/fwd v1.0.0 h1:UbZqGr5Y38ApvM/V/jEljVxwocdweyH+vmYvRPBnbqQ=
github.com/philhofer/fwd v1.0.0/go.mod h1:gk3iGcWd9+svBvR0sR+KPcfE+RNWozjowpeBVG3ZVNU=
github.com/pierrec/lz4 v2.0.5+incompatible h1:2xWsjqPFWcplujydGg4WmhC/6fZqK42wMM8aXeqhl0I=
github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pquerna/cachecontrol v0.0.0-20180517163645-1555304b9b35 h1:J9b7z+QKAmPf4YLrFg6oQUotqHQeUNWwkvo7jZp1GLU=
github.com/pquerna/cachecontrol v0.0.0-20180517163645-1555304b9b35/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
github.com/prometheus/client_golang v0.9.0-pre1 h1:AWTOhsOI9qxeirTuA0A4By/1Es1+y9EcCGY6bBZ2fhM=
github.com/prometheus/client_golang v0.9.0-pre1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4 h1:gQz4mCbXsO+nc9n1hCxHcGA3Zx3Eo+UHZoInFGUIXNM=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/common v0.7.0 h1:L+1lyG48J1zAQXA3RBX/nG/B3gjlHq0zTt2tlbJLyCY=
github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA=
github.com/prometheus/procfs v0.0.5 h1:3+auTFlqw+ZaQYJARz6ArODtkaIwtvBTx3N2NehQlL8=
github.com/prometheus/procfs v0.0.5/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ=
github.com/rifflock/lfshook v0.0.0-20180920164130-b9218ef580f5 h1:mZHayPoR0lNmnHyvtYjDeq0zlVHn9K/ZXoy17ylucdo=
github.com/rifflock/lfshook v0.0.0-20180920164130-b9218ef580f5/go.mod h1:GEXHk5HgEKCvEIIrSpFI3ozzG5xOKA2DVlEX/gGnewM=
github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/tinylib/msgp v1.1.0 h1:9fQd+ICuRIu/ue4vxJZu6/LzxN0HwMds2nq/0cFvxHU=
github.com/tinylib/msgp v1.1.0/go.mod h1:+d+yLhGm8mzTaHzB+wgMYrodPfmZrzkirds8fDWklFE=
github.com/xo/dburl v0.0.0-20191005012637-293c3298d6c0 h1:6DtWz8hNS4qbq0OCRPhdBMG9E2qKTSDKlwnP3dmZvuA=
github.com/xo/dburl v0.0.0-20191005012637-293c3298d6c0/go.mod h1:A47W3pdWONaZmXuLZgfKLAVgUY0qvfTRM5vVDKS40S4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190325154230-a5d413f7728c/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc h1:c0o/qxkaO2LF5t6fQrT4b5hzyggAkLLlCUjqfRxd8Q4=
golang.org/x/crypto v0.0.0-20191002192127-34f69633bfdc/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/net v0.0.0-20180218175443-cbe0f9307d01/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191007182048-72f939374954 h1:JGZucVF/L/TotR719NbujzadOZ2AgnYlqphQGHDCKaU=
golang.org/x/net v0.0.0-20191007182048-72f939374954/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be h1:vEDujvNQGv4jgYKudGeI/+DAX4Jffq6hpD55MmoEvKs=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191008105621-543471e840be h1:QAcqgptGM8IQBC9K/RC4o+O9YmqEm0diQn9QmZw/0mU=
golang.org/x/sys v0.0.0-20191008105621-543471e840be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.4.0 h1:/wp5JvzpHIxhs/dumFmF7BXTf3Z+dd4uXta4kVyO508=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20191007204434-a023cd5227bd h1:84VQPzup3IpKLxuIAZjHMhVjJ8fZ4/i3yUnj3k6fUdw=
google.golang.org/genproto v0.0.0-20191007204434-a023cd5227bd/go.mod h1:n3cpQtvxv34hfy77yVDNjmbRyujviMdxYliBSkLhpCc=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.24.0 h1:vb/1TCsVn3DcJlQ0Gs1yB1pKI6Do2/QNwxdKqmc/b0s=
google.golang.org/grpc v1.24.0/go.mod h1:XDChyiUovWa60DnaeDeZmSW86xtLtjtZbwvSiRnRtcA=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/coreos/go-oidc.v2 v2.1.0 h1:E8PjVFdj/SLDKB0hvb70KTbMbYVHjqztiQdSkIg8E+I=
gopkg.in/coreos/go-oidc.v2 v2.1.0/go.mod h1:fYaTe2FS96wZZwR17YTDHwG+Mw6fmyqJNxN2eNCGPCI=
gopkg.in/square/go-jose.v2 v2.4.0 h1:0kXPskUMGAXXWJlP05ktEMOV0vmzFQUWw6d+aZJQU8A=
gopkg.in/square/go-jose.v2 v2.4.0/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=
gopkg.in/urfave/cli.v2 v2.0.0-20180128181224-d604b6ffeee8 h1:/pLAskKF+d5SawboKd8GB8ew4ClHDbt2c3K9EBFeRGU=
gopkg.in/urfave/cli.v2 v2.0.0-20180128181224-d604b6ffeee8/go.mod h1:cKXr3E0k4aosgycml1b5z33BVV6hai1Kh7uDgFOkbcs=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4 h1:/eiJrUcujPVeJ3xlSWaiNi3uSVmDGBK1pDHUHAnao1I=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
zombiezen.com/go/capnproto2 v0.0.0-20180616160808-7cfd211c19c7 h1:CZoOFlTPbKfAShKYrMuUfYbnXexFT1rYRUX1SPnrdE4=
zombiezen.com/go/capnproto2 v0.0.0-20180616160808-7cfd211c19c7/go.mod h1:TMGa8HWGJkXiq4nHe9Zu/JgRF5oUtg4XizFC+Vexbec=
+13 -5
View File
@@ -3,6 +3,7 @@ package h2mux
import (
"sync"
"github.com/prometheus/client_golang/prometheus"
"golang.org/x/net/http2"
)
@@ -22,13 +23,16 @@ type activeStreamMap struct {
// ignoreNewStreams is true when the connection is being shut down. New streams
// cannot be registered.
ignoreNewStreams bool
// activeStreams is a gauge shared by all muxers of this process to expose the total number of active streams
activeStreams prometheus.Gauge
}
func newActiveStreamMap(useClientStreamNumbers bool) *activeStreamMap {
func newActiveStreamMap(useClientStreamNumbers bool, activeStreams prometheus.Gauge) *activeStreamMap {
m := &activeStreamMap{
streams: make(map[uint32]*MuxedStream),
streamsEmpty: make(chan struct{}),
nextStreamID: 1,
streams: make(map[uint32]*MuxedStream),
streamsEmpty: make(chan struct{}),
nextStreamID: 1,
activeStreams: activeStreams,
}
// Client initiated stream uses odd stream ID, server initiated stream uses even stream ID
if !useClientStreamNumbers {
@@ -63,6 +67,7 @@ func (m *activeStreamMap) Set(newStream *MuxedStream) bool {
return false
}
m.streams[newStream.streamID] = newStream
m.activeStreams.Inc()
return true
}
@@ -70,7 +75,10 @@ func (m *activeStreamMap) Set(newStream *MuxedStream) bool {
func (m *activeStreamMap) Delete(streamID uint32) {
m.Lock()
defer m.Unlock()
delete(m.streams, streamID)
if _, ok := m.streams[streamID]; ok {
delete(m.streams, streamID)
m.activeStreams.Dec()
}
if len(m.streams) == 0 && m.streamsEmpty != nil {
close(m.streamsEmpty)
m.streamsEmpty = nil
+13 -5
View File
@@ -7,6 +7,7 @@ import (
"sync"
"time"
"github.com/prometheus/client_golang/prometheus"
log "github.com/sirupsen/logrus"
"golang.org/x/net/http2"
"golang.org/x/net/http2/hpack"
@@ -19,7 +20,7 @@ const (
maxWindowSize uint32 = (1 << 31) - 1 // 2^31-1 = 2147483647, max window size in http2 spec
defaultTimeout time.Duration = 5 * time.Second
defaultRetries uint64 = 5
defaultWriteBufferMaxLen int = 1024 * 1024 * 512 // 500mb
defaultWriteBufferMaxLen int = 1024 * 1024 // 1mb
SettingMuxerMagic http2.SettingID = 0x42db
MuxerMagicOrigin uint32 = 0xa2e43c8b
@@ -107,6 +108,7 @@ func Handshake(
w io.WriteCloser,
r io.ReadCloser,
config MuxerConfig,
activeStreamsMetrics prometheus.Gauge,
) (*Muxer, error) {
// Set default config values
if config.Timeout == 0 {
@@ -130,7 +132,7 @@ func Handshake(
newStreamChan: make(chan MuxedStreamRequest),
abortChan: make(chan struct{}),
readyList: NewReadyList(),
streams: newActiveStreamMap(config.IsClient),
streams: newActiveStreamMap(config.IsClient, activeStreamsMetrics),
}
m.f.ReadMetaHeaders = hpack.NewDecoder(4096, func(hpack.HeaderField) {})
@@ -388,7 +390,7 @@ func isConnectionClosedError(err error) bool {
// Called by proxy server and tunnel
func (m *Muxer) OpenStream(ctx context.Context, headers []Header, body io.Reader) (*MuxedStream, error) {
stream := m.NewStream(headers)
if err := m.MakeMuxedStreamRequest(ctx, MuxedStreamRequest{stream, body}); err != nil {
if err := m.MakeMuxedStreamRequest(ctx, NewMuxedStreamRequest(stream, body)); err != nil {
return nil, err
}
if err := m.AwaitResponseHeaders(ctx, stream); err != nil {
@@ -397,10 +399,9 @@ func (m *Muxer) OpenStream(ctx context.Context, headers []Header, body io.Reader
return stream, nil
}
func (m *Muxer) OpenRPCStream(ctx context.Context) (*MuxedStream, error) {
stream := m.NewStream(RPCHeaders())
if err := m.MakeMuxedStreamRequest(ctx, MuxedStreamRequest{stream: stream, body: nil}); err != nil {
if err := m.MakeMuxedStreamRequest(ctx, NewMuxedStreamRequest(stream, nil)); err != nil {
return nil, err
}
if err := m.AwaitResponseHeaders(ctx, stream); err != nil {
@@ -425,6 +426,13 @@ func (m *Muxer) MakeMuxedStreamRequest(ctx context.Context, request MuxedStreamR
}
}
func (m *Muxer) CloseStreamRead(stream *MuxedStream) {
stream.CloseRead()
if stream.WriteClosed() {
m.streams.Delete(stream.streamID)
}
}
func (m *Muxer) AwaitResponseHeaders(ctx context.Context, stream *MuxedStream) error {
select {
case <-ctx.Done():
+20 -20
View File
@@ -43,7 +43,7 @@ type DefaultMuxerPair struct {
doneC chan struct{}
}
func NewDefaultMuxerPair(t assert.TestingT, f MuxedStreamFunc) *DefaultMuxerPair {
func NewDefaultMuxerPair(t assert.TestingT, testName string, f MuxedStreamFunc) *DefaultMuxerPair {
origin, edge := net.Pipe()
p := &DefaultMuxerPair{
OriginMuxConfig: MuxerConfig{
@@ -69,11 +69,11 @@ func NewDefaultMuxerPair(t assert.TestingT, f MuxedStreamFunc) *DefaultMuxerPair
EdgeConn: edge,
doneC: make(chan struct{}),
}
assert.NoError(t, p.Handshake())
assert.NoError(t, p.Handshake(testName))
return p
}
func NewCompressedMuxerPair(t assert.TestingT, quality CompressionSetting, f MuxedStreamFunc) *DefaultMuxerPair {
func NewCompressedMuxerPair(t assert.TestingT, testName string, quality CompressionSetting, f MuxedStreamFunc) *DefaultMuxerPair {
origin, edge := net.Pipe()
p := &DefaultMuxerPair{
OriginMuxConfig: MuxerConfig{
@@ -95,20 +95,20 @@ func NewCompressedMuxerPair(t assert.TestingT, quality CompressionSetting, f Mux
EdgeConn: edge,
doneC: make(chan struct{}),
}
assert.NoError(t, p.Handshake())
assert.NoError(t, p.Handshake(testName))
return p
}
func (p *DefaultMuxerPair) Handshake() error {
func (p *DefaultMuxerPair) Handshake(testName string) error {
ctx, cancel := context.WithTimeout(context.Background(), testHandshakeTimeout)
defer cancel()
errGroup, _ := errgroup.WithContext(ctx)
errGroup.Go(func() (err error) {
p.EdgeMux, err = Handshake(p.EdgeConn, p.EdgeConn, p.EdgeMuxConfig)
p.EdgeMux, err = Handshake(p.EdgeConn, p.EdgeConn, p.EdgeMuxConfig, NewActiveStreamsMetrics(testName, "edge"))
return errors.Wrap(err, "edge handshake failure")
})
errGroup.Go(func() (err error) {
p.OriginMux, err = Handshake(p.OriginConn, p.OriginConn, p.OriginMuxConfig)
p.OriginMux, err = Handshake(p.OriginConn, p.OriginConn, p.OriginMuxConfig, NewActiveStreamsMetrics(testName, "origin"))
return errors.Wrap(err, "origin handshake failure")
})
@@ -161,7 +161,7 @@ func TestHandshake(t *testing.T) {
f := func(stream *MuxedStream) error {
return nil
}
muxPair := NewDefaultMuxerPair(t, f)
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
AssertIfPipeReadable(t, muxPair.OriginConn)
AssertIfPipeReadable(t, muxPair.EdgeConn)
}
@@ -191,7 +191,7 @@ func TestSingleStream(t *testing.T) {
}
return nil
})
muxPair := NewDefaultMuxerPair(t, f)
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
muxPair.Serve(t)
stream, err := muxPair.OpenEdgeMuxStream(
@@ -262,7 +262,7 @@ func TestSingleStreamLargeResponseBody(t *testing.T) {
return nil
})
muxPair := NewDefaultMuxerPair(t, f)
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
muxPair.Serve(t)
stream, err := muxPair.OpenEdgeMuxStream(
@@ -309,7 +309,7 @@ func TestMultipleStreams(t *testing.T) {
log.Debugf("Wrote body for stream %s", stream.Headers[0].Value)
return nil
})
muxPair := NewDefaultMuxerPair(t, f)
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
muxPair.Serve(t)
maxStreams := 64
@@ -402,7 +402,7 @@ func TestMultipleStreamsFlowControl(t *testing.T) {
}
return nil
})
muxPair := NewDefaultMuxerPair(t, f)
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
muxPair.Serve(t)
errGroup, _ := errgroup.WithContext(context.Background())
@@ -461,7 +461,7 @@ func TestGracefulShutdown(t *testing.T) {
log.Debugf("Handler ends")
return nil
})
muxPair := NewDefaultMuxerPair(t, f)
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
muxPair.Serve(t)
stream, err := muxPair.OpenEdgeMuxStream(
@@ -516,7 +516,7 @@ func TestUnexpectedShutdown(t *testing.T) {
}
return nil
})
muxPair := NewDefaultMuxerPair(t, f)
muxPair := NewDefaultMuxerPair(t, t.Name(), f)
muxPair.Serve(t)
stream, err := muxPair.OpenEdgeMuxStream(
@@ -564,7 +564,7 @@ func EchoHandler(stream *MuxedStream) error {
func TestOpenAfterDisconnect(t *testing.T) {
for i := 0; i < 3; i++ {
muxPair := NewDefaultMuxerPair(t, EchoHandler)
muxPair := NewDefaultMuxerPair(t, fmt.Sprintf("%s_%d", t.Name(), i), EchoHandler)
muxPair.Serve(t)
switch i {
@@ -591,7 +591,7 @@ func TestOpenAfterDisconnect(t *testing.T) {
}
func TestHPACK(t *testing.T) {
muxPair := NewDefaultMuxerPair(t, EchoHandler)
muxPair := NewDefaultMuxerPair(t, t.Name(), EchoHandler)
muxPair.Serve(t)
stream, err := muxPair.OpenEdgeMuxStream(
@@ -724,7 +724,7 @@ func TestMultipleStreamsWithDictionaries(t *testing.T) {
return nil
})
muxPair := NewCompressedMuxerPair(t, q, f)
muxPair := NewCompressedMuxerPair(t, fmt.Sprintf("%s_%d", t.Name(), q), q, f)
muxPair.Serve(t)
var wg sync.WaitGroup
@@ -918,7 +918,7 @@ func TestSampleSiteWithDictionaries(t *testing.T) {
assert.NoError(t, err)
for q := CompressionNone; q <= CompressionMax; q++ {
muxPair := NewCompressedMuxerPair(t, q, sampleSiteHandler(files))
muxPair := NewCompressedMuxerPair(t, fmt.Sprintf("%s_%d", t.Name(), q), q, sampleSiteHandler(files))
muxPair.Serve(t)
var wg sync.WaitGroup
@@ -957,7 +957,7 @@ func TestLongSiteWithDictionaries(t *testing.T) {
files, err := loadSampleFiles(paths)
assert.NoError(t, err)
for q := CompressionNone; q <= CompressionMedium; q++ {
muxPair := NewCompressedMuxerPair(t, q, sampleSiteHandler(files))
muxPair := NewCompressedMuxerPair(t, fmt.Sprintf("%s_%d", t.Name(), q), q, sampleSiteHandler(files))
muxPair.Serve(t)
rand.Seed(time.Now().Unix())
@@ -998,7 +998,7 @@ func BenchmarkOpenStream(b *testing.B) {
})
return nil
})
muxPair := NewDefaultMuxerPair(b, f)
muxPair := NewDefaultMuxerPair(b, fmt.Sprintf("%s_%d", b.Name(), i), f)
muxPair.Serve(b)
b.StartTimer()
openStreams(b, muxPair, streams)
+32 -9
View File
@@ -137,9 +137,10 @@ func (s *MuxedStream) Write(p []byte) (int, error) {
// If the buffer is full, block till there is more room.
// Use a loop to recheck the buffer size after the lock is reacquired.
for s.writeBufferMaxLen <= s.writeBuffer.Len() {
s.writeLock.Unlock()
<-s.writeBufferHasSpace
s.writeLock.Lock()
s.awaitWriteBufferHasSpace()
if s.writeEOF {
return totalWritten, io.EOF
}
}
amountToWrite := len(p) - totalWritten
spaceAvailable := s.writeBufferMaxLen - s.writeBuffer.Len()
@@ -188,10 +189,19 @@ func (s *MuxedStream) CloseWrite() error {
if c, ok := s.writeBuffer.(io.Closer); ok {
c.Close()
}
// Allow MuxedStream::Write() to terminate its loop with err=io.EOF, if needed
s.notifyWriteBufferHasSpace()
// We need to send something over the wire, even if it's an END_STREAM with no data
s.writeNotify()
return nil
}
func (s *MuxedStream) WriteClosed() bool {
s.writeLock.Lock()
defer s.writeLock.Unlock()
return s.writeEOF
}
func (s *MuxedStream) WriteHeaders(headers []Header) error {
s.writeLock.Lock()
defer s.writeLock.Unlock()
@@ -232,6 +242,23 @@ func (s *MuxedStream) TunnelHostname() TunnelHostname {
return s.tunnelHostname
}
// Block until a value is sent on writeBufferHasSpace.
// Must be called while holding writeLock
func (s *MuxedStream) awaitWriteBufferHasSpace() {
s.writeLock.Unlock()
<-s.writeBufferHasSpace
s.writeLock.Lock()
}
// Send a value on writeBufferHasSpace without blocking.
// Must be called while holding writeLock
func (s *MuxedStream) notifyWriteBufferHasSpace() {
select {
case s.writeBufferHasSpace <- struct{}{}:
default:
}
}
func (s *MuxedStream) getReceiveWindow() uint32 {
s.writeLock.Lock()
defer s.writeLock.Unlock()
@@ -351,17 +378,13 @@ func (s *MuxedStream) getChunk() *streamChunk {
sendData: !s.sentEOF,
eof: s.writeEOF && uint32(s.writeBuffer.Len()) <= s.sendWindow,
}
// Copy at most s.sendWindow bytes, adjust the sendWindow accordingly
writeLen, _ := io.CopyN(&chunk.buffer, s.writeBuffer, int64(s.sendWindow))
s.sendWindow -= uint32(writeLen)
// Non-blocking channel send. This will allow MuxedStream::Write() to continue, if needed
// Allow MuxedStream::Write() to continue, if needed
if s.writeBuffer.Len() < s.writeBufferMaxLen {
select {
case s.writeBufferHasSpace <- struct{}{}:
default:
}
s.notifyWriteBufferHasSpace()
}
// When we write the chunk, we'll write the WINDOW_UPDATE frame if needed
+12
View File
@@ -5,6 +5,7 @@ import (
"time"
"github.com/golang-collections/collections/queue"
"github.com/prometheus/client_golang/prometheus"
log "github.com/sirupsen/logrus"
)
@@ -299,3 +300,14 @@ func (r *rate) get() (curr, min, max uint64) {
defer r.lock.RUnlock()
return r.curr, r.min, r.max
}
func NewActiveStreamsMetrics(namespace, subsystem string) prometheus.Gauge {
activeStreams := prometheus.NewGauge(prometheus.GaugeOpts{
Namespace: namespace,
Subsystem: subsystem,
Name: "active_streams",
Help: "Number of active streams created by all muxers.",
})
prometheus.MustRegister(activeStreams)
return activeStreams
}
+12 -4
View File
@@ -87,23 +87,28 @@ func (r *MuxReader) run(parentLogger *log.Entry) error {
for {
frame, err := r.f.ReadFrame()
if err != nil {
errLogger := logger.WithError(err)
if errorDetail := r.f.ErrorDetail(); errorDetail != nil {
errLogger = errLogger.WithField("errorDetail", errorDetail)
}
switch e := err.(type) {
case http2.StreamError:
logger.WithError(err).Warn("stream error")
errLogger.Warn("stream error")
r.streamError(e.StreamID, e.Code)
case http2.ConnectionError:
logger.WithError(err).Warn("connection error")
errLogger.Warn("connection error")
return r.connectionError(err)
default:
if isConnectionClosedError(err) {
if r.streams.Len() == 0 {
// don't log the error here -- that would just be extra noise
logger.Debug("shutting down")
return nil
}
logger.Warn("connection closed unexpectedly")
errLogger.Warn("connection closed unexpectedly")
return err
} else {
logger.WithError(err).Warn("frame read error")
errLogger.Warn("frame read error")
return r.connectionError(err)
}
}
@@ -120,6 +125,9 @@ func (r *MuxReader) run(parentLogger *log.Entry) error {
if streamID == 0 {
return ErrInvalidStream
}
if stream, ok := r.streams.Get(streamID); ok {
stream.Close()
}
r.streams.Delete(streamID)
case *http2.PingFrame:
r.receivePingData(f)
+2 -2
View File
@@ -59,7 +59,7 @@ func assertOpenStreamSucceed(t *testing.T, stream *MuxedStream, err error) {
func TestMissingHeaders(t *testing.T) {
originHandler := &mockOriginStreamHandler{}
muxPair := NewDefaultMuxerPair(t, originHandler.ServeStream)
muxPair := NewDefaultMuxerPair(t, t.Name(), originHandler.ServeStream)
muxPair.Serve(t)
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
@@ -83,7 +83,7 @@ func TestMissingHeaders(t *testing.T) {
func TestReceiveHeaderData(t *testing.T) {
originHandler := &mockOriginStreamHandler{}
muxPair := NewDefaultMuxerPair(t, originHandler.ServeStream)
muxPair := NewDefaultMuxerPair(t, t.Name(), originHandler.ServeStream)
muxPair.Serve(t)
reqHeaders := []Header{
+7
View File
@@ -54,6 +54,13 @@ type MuxedStreamRequest struct {
body io.Reader
}
func NewMuxedStreamRequest(stream *MuxedStream, body io.Reader) MuxedStreamRequest {
return MuxedStreamRequest{
stream: stream,
body: body,
}
}
func (r *MuxedStreamRequest) flushBody() {
io.Copy(r.stream, r.body)
r.stream.CloseWrite()
+5 -4
View File
@@ -103,15 +103,16 @@ func StartHelloWorldServer(logger *logrus.Logger, listener net.Listener, shutdow
WriteBufferSize: 1024,
}
httpServer := &http.Server{Addr: listener.Addr().String(), Handler: nil}
muxer := http.NewServeMux()
muxer.HandleFunc("/uptime", uptimeHandler(time.Now()))
muxer.HandleFunc("/ws", websocketHandler(logger, upgrader))
muxer.HandleFunc("/", rootHandler(serverName))
httpServer := &http.Server{Addr: listener.Addr().String(), Handler: muxer}
go func() {
<-shutdownC
httpServer.Close()
}()
http.HandleFunc("/uptime", uptimeHandler(time.Now()))
http.HandleFunc("/ws", websocketHandler(logger, upgrader))
http.HandleFunc("/", rootHandler(serverName))
err := httpServer.Serve(listener)
return err
}
+4
View File
@@ -2,6 +2,7 @@ package metrics
import (
"context"
"fmt"
"net"
"net/http"
_ "net/http/pprof"
@@ -33,6 +34,9 @@ func ServeMetrics(l net.Listener, shutdownC <-chan struct{}, logger *logrus.Logg
}
http.Handle("/metrics", promhttp.Handler())
http.Handle("/healthcheck", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "OK\n")
}))
wg.Add(1)
go func() {
+134 -62
View File
@@ -9,6 +9,12 @@ import (
"github.com/prometheus/client_golang/prometheus"
)
const (
metricsNamespace = "cloudflared"
tunnelSubsystem = "tunnel"
muxerSubsystem = "muxer"
)
type muxerMetrics struct {
rtt *prometheus.GaugeVec
rttMin *prometheus.GaugeVec
@@ -32,6 +38,7 @@ type muxerMetrics struct {
type TunnelMetrics struct {
haConnections prometheus.Gauge
activeStreams prometheus.Gauge
totalRequests prometheus.Counter
requestsPerTunnel *prometheus.CounterVec
// concurrentRequestsLock is a mutex for concurrentRequests and maxConcurrentRequests
@@ -63,8 +70,10 @@ type TunnelMetrics struct {
func newMuxerMetrics() *muxerMetrics {
rtt := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "rtt",
Help: "Round-trip time in millisecond",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "rtt",
Help: "Round-trip time in millisecond",
},
[]string{"connection_id"},
)
@@ -72,8 +81,10 @@ func newMuxerMetrics() *muxerMetrics {
rttMin := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "rtt_min",
Help: "Shortest round-trip time in millisecond",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "rtt_min",
Help: "Shortest round-trip time in millisecond",
},
[]string{"connection_id"},
)
@@ -81,8 +92,10 @@ func newMuxerMetrics() *muxerMetrics {
rttMax := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "rtt_max",
Help: "Longest round-trip time in millisecond",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "rtt_max",
Help: "Longest round-trip time in millisecond",
},
[]string{"connection_id"},
)
@@ -90,8 +103,10 @@ func newMuxerMetrics() *muxerMetrics {
receiveWindowAve := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "receive_window_ave",
Help: "Average receive window size in bytes",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "receive_window_ave",
Help: "Average receive window size in bytes",
},
[]string{"connection_id"},
)
@@ -99,8 +114,10 @@ func newMuxerMetrics() *muxerMetrics {
sendWindowAve := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "send_window_ave",
Help: "Average send window size in bytes",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "send_window_ave",
Help: "Average send window size in bytes",
},
[]string{"connection_id"},
)
@@ -108,8 +125,10 @@ func newMuxerMetrics() *muxerMetrics {
receiveWindowMin := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "receive_window_min",
Help: "Smallest receive window size in bytes",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "receive_window_min",
Help: "Smallest receive window size in bytes",
},
[]string{"connection_id"},
)
@@ -117,8 +136,10 @@ func newMuxerMetrics() *muxerMetrics {
receiveWindowMax := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "receive_window_max",
Help: "Largest receive window size in bytes",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "receive_window_max",
Help: "Largest receive window size in bytes",
},
[]string{"connection_id"},
)
@@ -126,8 +147,10 @@ func newMuxerMetrics() *muxerMetrics {
sendWindowMin := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "send_window_min",
Help: "Smallest send window size in bytes",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "send_window_min",
Help: "Smallest send window size in bytes",
},
[]string{"connection_id"},
)
@@ -135,8 +158,10 @@ func newMuxerMetrics() *muxerMetrics {
sendWindowMax := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "send_window_max",
Help: "Largest send window size in bytes",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "send_window_max",
Help: "Largest send window size in bytes",
},
[]string{"connection_id"},
)
@@ -144,8 +169,10 @@ func newMuxerMetrics() *muxerMetrics {
inBoundRateCurr := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "inbound_bytes_per_sec_curr",
Help: "Current inbounding bytes per second, 0 if there is no incoming connection",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "inbound_bytes_per_sec_curr",
Help: "Current inbounding bytes per second, 0 if there is no incoming connection",
},
[]string{"connection_id"},
)
@@ -153,8 +180,10 @@ func newMuxerMetrics() *muxerMetrics {
inBoundRateMin := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "inbound_bytes_per_sec_min",
Help: "Minimum non-zero inbounding bytes per second",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "inbound_bytes_per_sec_min",
Help: "Minimum non-zero inbounding bytes per second",
},
[]string{"connection_id"},
)
@@ -162,8 +191,10 @@ func newMuxerMetrics() *muxerMetrics {
inBoundRateMax := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "inbound_bytes_per_sec_max",
Help: "Maximum inbounding bytes per second",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "inbound_bytes_per_sec_max",
Help: "Maximum inbounding bytes per second",
},
[]string{"connection_id"},
)
@@ -171,8 +202,10 @@ func newMuxerMetrics() *muxerMetrics {
outBoundRateCurr := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "outbound_bytes_per_sec_curr",
Help: "Current outbounding bytes per second, 0 if there is no outgoing traffic",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "outbound_bytes_per_sec_curr",
Help: "Current outbounding bytes per second, 0 if there is no outgoing traffic",
},
[]string{"connection_id"},
)
@@ -180,8 +213,10 @@ func newMuxerMetrics() *muxerMetrics {
outBoundRateMin := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "outbound_bytes_per_sec_min",
Help: "Minimum non-zero outbounding bytes per second",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "outbound_bytes_per_sec_min",
Help: "Minimum non-zero outbounding bytes per second",
},
[]string{"connection_id"},
)
@@ -189,8 +224,10 @@ func newMuxerMetrics() *muxerMetrics {
outBoundRateMax := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "outbound_bytes_per_sec_max",
Help: "Maximum outbounding bytes per second",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "outbound_bytes_per_sec_max",
Help: "Maximum outbounding bytes per second",
},
[]string{"connection_id"},
)
@@ -198,8 +235,10 @@ func newMuxerMetrics() *muxerMetrics {
compBytesBefore := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "comp_bytes_before",
Help: "Bytes sent via cross-stream compression, pre compression",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "comp_bytes_before",
Help: "Bytes sent via cross-stream compression, pre compression",
},
[]string{"connection_id"},
)
@@ -207,8 +246,10 @@ func newMuxerMetrics() *muxerMetrics {
compBytesAfter := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "comp_bytes_after",
Help: "Bytes sent via cross-stream compression, post compression",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "comp_bytes_after",
Help: "Bytes sent via cross-stream compression, post compression",
},
[]string{"connection_id"},
)
@@ -216,8 +257,10 @@ func newMuxerMetrics() *muxerMetrics {
compRateAve := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "comp_rate_ave",
Help: "Average outbound cross-stream compression ratio",
Namespace: metricsNamespace,
Subsystem: muxerSubsystem,
Name: "comp_rate_ave",
Help: "Average outbound cross-stream compression ratio",
},
[]string{"connection_id"},
)
@@ -274,22 +317,30 @@ func convertRTTMilliSec(t time.Duration) float64 {
func NewTunnelMetrics() *TunnelMetrics {
haConnections := prometheus.NewGauge(
prometheus.GaugeOpts{
Name: "ha_connections",
Help: "Number of active ha connections",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "ha_connections",
Help: "Number of active ha connections",
})
prometheus.MustRegister(haConnections)
activeStreams := h2mux.NewActiveStreamsMetrics(metricsNamespace, tunnelSubsystem)
totalRequests := prometheus.NewCounter(
prometheus.CounterOpts{
Name: "total_requests",
Help: "Amount of requests proxied through all the tunnels",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "total_requests",
Help: "Amount of requests proxied through all the tunnels",
})
prometheus.MustRegister(totalRequests)
requestsPerTunnel := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "requests_per_tunnel",
Help: "Amount of requests proxied through each tunnel",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "requests_per_tunnel",
Help: "Amount of requests proxied through each tunnel",
},
[]string{"connection_id"},
)
@@ -297,8 +348,10 @@ func NewTunnelMetrics() *TunnelMetrics {
concurrentRequestsPerTunnel := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "concurrent_requests_per_tunnel",
Help: "Concurrent requests proxied through each tunnel",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "concurrent_requests_per_tunnel",
Help: "Concurrent requests proxied through each tunnel",
},
[]string{"connection_id"},
)
@@ -306,8 +359,10 @@ func NewTunnelMetrics() *TunnelMetrics {
maxConcurrentRequestsPerTunnel := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "max_concurrent_requests_per_tunnel",
Help: "Largest number of concurrent requests proxied through each tunnel so far",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "max_concurrent_requests_per_tunnel",
Help: "Largest number of concurrent requests proxied through each tunnel so far",
},
[]string{"connection_id"},
)
@@ -315,15 +370,19 @@ func NewTunnelMetrics() *TunnelMetrics {
timerRetries := prometheus.NewGauge(
prometheus.GaugeOpts{
Name: "timer_retries",
Help: "Unacknowledged heart beats count",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "timer_retries",
Help: "Unacknowledged heart beats count",
})
prometheus.MustRegister(timerRetries)
responseByCode := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "response_by_code",
Help: "Count of responses by HTTP status code",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "response_by_code",
Help: "Count of responses by HTTP status code",
},
[]string{"status_code"},
)
@@ -331,8 +390,10 @@ func NewTunnelMetrics() *TunnelMetrics {
responseCodePerTunnel := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "response_code_per_tunnel",
Help: "Count of responses by HTTP status code fore each tunnel",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "response_code_per_tunnel",
Help: "Count of responses by HTTP status code fore each tunnel",
},
[]string{"connection_id", "status_code"},
)
@@ -340,8 +401,10 @@ func NewTunnelMetrics() *TunnelMetrics {
serverLocations := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "server_locations",
Help: "Where each tunnel is connected to. 1 means current location, 0 means previous locations.",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "server_locations",
Help: "Where each tunnel is connected to. 1 means current location, 0 means previous locations.",
},
[]string{"connection_id", "location"},
)
@@ -349,8 +412,10 @@ func NewTunnelMetrics() *TunnelMetrics {
rpcFail := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "tunnel_rpc_fail",
Help: "Count of RPC connection errors by type",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "tunnel_rpc_fail",
Help: "Count of RPC connection errors by type",
},
[]string{"error"},
)
@@ -358,8 +423,10 @@ func NewTunnelMetrics() *TunnelMetrics {
registerFail := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "tunnel_register_fail",
Help: "Count of tunnel registration errors by type",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "tunnel_register_fail",
Help: "Count of tunnel registration errors by type",
},
[]string{"error"},
)
@@ -367,8 +434,10 @@ func NewTunnelMetrics() *TunnelMetrics {
userHostnamesCounts := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "user_hostnames_counts",
Help: "Which user hostnames cloudflared is serving",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "user_hostnames_counts",
Help: "Which user hostnames cloudflared is serving",
},
[]string{"userHostname"},
)
@@ -376,13 +445,16 @@ func NewTunnelMetrics() *TunnelMetrics {
registerSuccess := prometheus.NewCounter(
prometheus.CounterOpts{
Name: "tunnel_register_success",
Help: "Count of successful tunnel registrations",
Namespace: metricsNamespace,
Subsystem: tunnelSubsystem,
Name: "tunnel_register_success",
Help: "Count of successful tunnel registrations",
})
prometheus.MustRegister(registerSuccess)
return &TunnelMetrics{
haConnections: haConnections,
activeStreams: activeStreams,
totalRequests: totalRequests,
requestsPerTunnel: requestsPerTunnel,
concurrentRequestsPerTunnel: concurrentRequestsPerTunnel,
+25 -17
View File
@@ -23,7 +23,6 @@ import (
"github.com/cloudflare/cloudflared/validation"
"github.com/cloudflare/cloudflared/websocket"
raven "github.com/getsentry/raven-go"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/prometheus/client_golang/prometheus"
@@ -152,7 +151,7 @@ func ServeTunnelLoop(ctx context.Context,
connectedSignal *signal.Signal,
u uuid.UUID,
) error {
logger := config.Logger
connectionLogger := config.Logger.WithField("connectionID", connectionID)
config.Metrics.incrementHaConnections()
defer config.Metrics.decrementHaConnections()
backoff := BackoffHandler{MaxRetries: config.Retries}
@@ -165,10 +164,18 @@ func ServeTunnelLoop(ctx context.Context,
// Ensure the above goroutine will terminate if we return without connecting
defer connectedFuse.Fuse(false)
for {
err, recoverable := ServeTunnel(ctx, config, addr, connectionID, connectedFuse, &backoff, u)
err, recoverable := ServeTunnel(
ctx,
config,
connectionLogger,
addr, connectionID,
connectedFuse,
&backoff,
u,
)
if recoverable {
if duration, ok := backoff.GetBackoffDuration(ctx); ok {
logger.Infof("Retrying in %s seconds", duration)
connectionLogger.Infof("Retrying in %s seconds", duration)
backoff.Backoff(ctx)
continue
}
@@ -180,6 +187,7 @@ func ServeTunnelLoop(ctx context.Context,
func ServeTunnel(
ctx context.Context,
config *TunnelConfig,
logger *log.Entry,
addr *net.TCPAddr,
connectionID uint8,
connectedFuse *h2mux.BooleanFuse,
@@ -199,7 +207,6 @@ func ServeTunnel(
}()
connectionTag := uint8ToString(connectionID)
logger := config.Logger.WithField("connectionID", connectionTag)
// additional tags to send other than hostname which is set in cloudflared main package
tags := make(map[string]string)
@@ -208,7 +215,7 @@ func ServeTunnel(
// Returns error from parsing the origin URL or handshake errors
handler, originLocalIP, err := NewTunnelHandler(ctx, config, addr.String(), connectionID)
if err != nil {
errLog := config.Logger.WithError(err)
errLog := logger.WithError(err)
switch err.(type) {
case dialError:
errLog.Error("Unable to dial edge")
@@ -224,7 +231,7 @@ func ServeTunnel(
errGroup, serveCtx := errgroup.WithContext(ctx)
errGroup.Go(func() error {
err := RegisterTunnel(serveCtx, handler.muxer, config, connectionID, originLocalIP, u)
err := RegisterTunnel(serveCtx, handler.muxer, config, logger, connectionID, originLocalIP, u)
if err == nil {
connectedFuse.Fuse(true)
backoff.SetGracePeriod()
@@ -260,6 +267,8 @@ func ServeTunnel(
err = errGroup.Wait()
if err != nil {
_ = newClientRegisterTunnelError(err, config.Metrics.regFail)
switch castedErr := err.(type) {
case dupConnRegisterTunnelError:
logger.Info("Already connected to this server, selecting a different one")
@@ -274,14 +283,12 @@ func ServeTunnel(
return castedErr.cause, !castedErr.permanent
case clientRegisterTunnelError:
logger.WithError(castedErr.cause).Error("Register tunnel error on client side")
raven.CaptureError(castedErr.cause, tags)
return err, true
case muxerShutdownError:
logger.Infof("Muxer shutdown")
return err, true
default:
logger.WithError(err).Error("Serve tunnel error")
raven.CaptureError(err, tags)
return err, true
}
}
@@ -302,6 +309,7 @@ func RegisterTunnel(
ctx context.Context,
muxer *h2mux.Muxer,
config *TunnelConfig,
logger *log.Entry,
connectionID uint8,
originLocalIP string,
uuid uuid.UUID,
@@ -333,13 +341,13 @@ func RegisterTunnel(
config.Hostname,
config.RegistrationOptions(connectionID, originLocalIP, uuid),
)
LogServerInfo(serverInfoPromise.Result(), connectionID, config.Metrics, config.Logger)
LogServerInfo(serverInfoPromise.Result(), connectionID, config.Metrics, logger)
if err != nil {
// RegisterTunnel RPC failure
return newClientRegisterTunnelError(err, config.Metrics.regFail)
}
for _, logLine := range registration.LogLines {
config.Logger.Info(logLine)
logger.Info(logLine)
}
if regErr := processRegisterTunnelError(registration.Err, registration.PermanentFailure, config.Metrics); regErr != nil {
@@ -348,24 +356,24 @@ func RegisterTunnel(
if registration.TunnelID != "" {
config.Metrics.tunnelsHA.AddTunnelID(connectionID, registration.TunnelID)
config.Logger.Infof("Each HA connection's tunnel IDs: %v", config.Metrics.tunnelsHA.String())
logger.Infof("Each HA connection's tunnel IDs: %v", config.Metrics.tunnelsHA.String())
}
// Print out the user's trial zone URL in a nice box (if they requested and got one)
if isTrialTunnel := config.Hostname == ""; isTrialTunnel {
if url, err := url.Parse(registration.Url); err == nil {
for _, line := range asciiBox(trialZoneMsg(url.String()), 2) {
config.Logger.Infoln(line)
logger.Infoln(line)
}
} else {
config.Logger.Errorln("Failed to connect tunnel, please try again.")
logger.Errorln("Failed to connect tunnel, please try again.")
return fmt.Errorf("empty URL in response from Cloudflare edge")
}
}
config.Metrics.userHostnamesCounts.WithLabelValues(registration.Url).Inc()
config.Logger.Infof("Route propagating, it may take up to 1 minute for your new route to become functional")
logger.Infof("Route propagating, it may take up to 1 minute for your new route to become functional")
return nil
}
@@ -421,7 +429,7 @@ func LogServerInfo(
promise tunnelrpc.ServerInfo_Promise,
connectionID uint8,
metrics *TunnelMetrics,
logger *log.Logger,
logger *log.Entry,
) {
serverInfoMessage, err := promise.Struct()
if err != nil {
@@ -513,7 +521,7 @@ func NewTunnelHandler(ctx context.Context,
MaxHeartbeats: config.MaxHeartbeats,
Logger: config.TransportLogger.WithFields(log.Fields{}),
CompressionQuality: h2mux.CompressionSetting(config.CompressionQuality),
})
}, h.metrics.activeStreams)
if err != nil {
return h, "", errors.New("TLS handshake error")
}
-31
View File
@@ -1,31 +0,0 @@
#!/bin/bash
FILENAME=$1
VERSION=$2
TAP_ROOT=$3
URL="https://developers.cloudflare.com/argo-tunnel/dl/cloudflared-${VERSION}-darwin-amd64.tgz"
SHA256=$(sha256sum -b "${FILENAME}" | cut -b1-64)
cd "${TAP_ROOT}" || exit 1
git checkout -f master
git reset --hard origin/master
tee cloudflared.rb <<EOF
class Cloudflared < Formula
desc 'Argo Tunnel'
homepage 'https://developers.cloudflare.com/argo-tunnel/'
url '${URL}'
sha256 '${SHA256}'
version '${VERSION}'
def install
bin.install 'cloudflared'
end
end
EOF
git add cloudflared.rb
git config user.name "cloudflare-warp-bot"
git config user.email "warp-bot@cloudflare.com"
git commit -m "Release Argo Tunnel ${VERSION}"
git version
GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=../github_known_hosts" git push -v origin master
+14
View File
@@ -0,0 +1,14 @@
FROM python:3-buster
RUN wget https://bin.equinox.io/c/VdrWdbjqyF/cloudflared-stable-linux-amd64.deb \
&& dpkg -i cloudflared-stable-linux-amd64.deb
RUN pip install pexpect
COPY tests.py .
COPY ssh /root/.ssh
RUN chmod 600 /root/.ssh/id_rsa
ARG SSH_HOSTNAME
RUN bash -c 'sed -i "s/{{hostname}}/${SSH_HOSTNAME}/g" /root/.ssh/authorized_keys_config'
RUN bash -c 'sed -i "s/{{hostname}}/${SSH_HOSTNAME}/g" /root/.ssh/short_lived_cert_config'
+23
View File
@@ -0,0 +1,23 @@
# Cloudflared SSH server smoke tests
Runs several tests in a docker container against a server that is started out of band of these tests.
Cloudflared token also needs to be retrieved out of band.
SSH server hostname and user need to be configured in a docker environment file
## Running tests
* Build cloudflared:
make cloudflared
* Start server:
sudo ./cloudflared tunnel --hostname HOSTNAME --ssh-server
* Fetch token:
./cloudflared access login HOSTNAME
* Create docker env file:
echo "SSH_HOSTNAME=HOSTNAME\nSSH_USER=USERNAME\n" > ssh_server_tests/.env
* Run tests:
make test-ssh-server
+18
View File
@@ -0,0 +1,18 @@
version: "3.1"
services:
ssh_test:
build:
context: .
args:
- SSH_HOSTNAME=${SSH_HOSTNAME}
volumes:
- "~/.cloudflared/:/root/.cloudflared"
env_file:
- .env
environment:
- AUTHORIZED_KEYS_SSH_CONFIG=/root/.ssh/authorized_keys_config
- SHORT_LIVED_CERT_SSH_CONFIG=/root/.ssh/short_lived_cert_config
- REMOTE_SCP_FILENAME=scp_test.txt
- ROOT_ONLY_TEST_FILE_PATH=~/permission_test.txt
entrypoint: "python tests.py"
@@ -0,0 +1,5 @@
Host *
AddressFamily inet
Host {{hostname}}
ProxyCommand /usr/local/bin/cloudflared access ssh --hostname %h
+49
View File
@@ -0,0 +1,49 @@
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAACFwAAAAdzc2gtcn
NhAAAAAwEAAQAAAgEAvi26NDQ8cYTTztqPe9ZgF5HR/rIo5FoDgL5NbbZKW6h0txP9Fd8s
id9Bgmo+aGkeM327tPVVMQ6UFmdRksOCIDWQDjNLF8b6S+Fu95tvMKSbGreRoR32OvgZKV
I6KmOsF4z4GIv9naPplZswtKEUhSSI+/gPdAs9wfwalqZ77e82QJ727bYMeC3lzuoT+KBI
dYufJ4OQhLtpHrqhB5sn7s6+oCv/u85GSln5SIC18Hi2t9lW4tgb5tH8P0kEDDWGfPS5ok
qGi4kFTvwBXOCS2r4dhi5hRkpP7PqG4np0OCfvK5IRRJ27fCnj0loc+puZJAxnPMbuJr64
vwxRx78PM/V0PDUsl0P6aR/vbe0XmF9FGqbWf2Tar1p4r6C9/bMzcDz8seYT8hzLIHP3+R
l1hdlsTLm+1EzhaExKId+tjXegKGG4nU24h6qHEnRxLQDMwEsdkfj4E1pVypZJXVyNj99D
o84vi0EUnu7R4HmQb/C+Pu7qMDtLT3Zk7O5Mg4LQ+cTz9V0noYEAyG46nAB4U/nJzBnV1J
+aAdpioHmUAYhLYlQ9Kiy7LCJi92g9Wqa4wxMKxBbO5ZeH++p2p2lUi/oQNqx/2dLYFmy0
wxvJHbZIhAaFbOeCvHg1ucIAQznli2jOr2qoB+yKRRPAp/3NXnZg1v7ce2CkwiAD52wjtC
kAAAdILMJUeyzCVHsAAAAHc3NoLXJzYQAAAgEAvi26NDQ8cYTTztqPe9ZgF5HR/rIo5FoD
gL5NbbZKW6h0txP9Fd8sid9Bgmo+aGkeM327tPVVMQ6UFmdRksOCIDWQDjNLF8b6S+Fu95
tvMKSbGreRoR32OvgZKVI6KmOsF4z4GIv9naPplZswtKEUhSSI+/gPdAs9wfwalqZ77e82
QJ727bYMeC3lzuoT+KBIdYufJ4OQhLtpHrqhB5sn7s6+oCv/u85GSln5SIC18Hi2t9lW4t
gb5tH8P0kEDDWGfPS5okqGi4kFTvwBXOCS2r4dhi5hRkpP7PqG4np0OCfvK5IRRJ27fCnj
0loc+puZJAxnPMbuJr64vwxRx78PM/V0PDUsl0P6aR/vbe0XmF9FGqbWf2Tar1p4r6C9/b
MzcDz8seYT8hzLIHP3+Rl1hdlsTLm+1EzhaExKId+tjXegKGG4nU24h6qHEnRxLQDMwEsd
kfj4E1pVypZJXVyNj99Do84vi0EUnu7R4HmQb/C+Pu7qMDtLT3Zk7O5Mg4LQ+cTz9V0noY
EAyG46nAB4U/nJzBnV1J+aAdpioHmUAYhLYlQ9Kiy7LCJi92g9Wqa4wxMKxBbO5ZeH++p2
p2lUi/oQNqx/2dLYFmy0wxvJHbZIhAaFbOeCvHg1ucIAQznli2jOr2qoB+yKRRPAp/3NXn
Zg1v7ce2CkwiAD52wjtCkAAAADAQABAAACAQCbnVsyAFQ9J00Rg/HIiUATyTQlzq57O9SF
8jH1RiZOHedzLx32WaleH5rBFiJ+2RTnWUjQ57aP77fpJR2wk93UcT+w/vPBPwXsNUjRvx
Qan3ZzRCYbyiKDWiNslmYV7X0RwD36CAK8jTVDP7t48h2SXLTiSLaMY+5i3uD6yLu7k/O2
qNyw4jgN1rCmwQ8acD0aQec3NAZ7NcbsaBX/3Uutsup0scwOZtlJWZoLY5Z8cKpCgcsAz4
j1NHnNZvey7dFgSffj/ktdvf7kBH0w/GnuJ4aNF0Jte70u0kiw5TZYBQVFh74tgUu6a6SJ
qUbxIYUL5EJNjxGsDn+phHEemw3aMv0CwZG6Tqaionlna7bLsl9Bg1HTGclczVWx8uqC+M
6agLmkhYCHG0rVj8h5smjXAQXtmvIDVYDOlJZZoF9VAOCj6QfmJUH1NAGpCs1HDHbeOxGA
OLCh4d3F4rScPqhGdtSt4W13VFIvXn2Qqoz9ufepZsee1SZqpcerxywx2wN9ZAzu+X8lTN
i+TA2B3vWpqqucOEsp4JwDN+VMKZqKUGUDWcm/eHSaG6wq0q734LUlgM85TjaIg8QsNtWV
giB1nWwsYIuH4rsFNFGEwURYdGBcw6idH0GZ7I4RaIB5F9oOza1d601E0APHYrtnx9yOiK
nOtJ+5ZmVZovaDRfu1aQAAAQBU/EFaNUzoVhO04pS2L6BlByt963bOIsSJhdlEzek5AAli
eaf1S/PD6xWCc0IGY+GZE0HPbhsKYanjqOpWldcA2T7fzf4oz4vFBfUkPYo/MLSlLCYsDd
IH3wBkCssnfR5EkzNgxnOvq646Nl64BMvxwSIXGPktdq9ZALxViwricSRzCFURnh5vLHWU
wBzSgAA0UlZ9E64GtAv066+AoZCp83GhTLRC4o0naE2e/K4op4BCFHLrZ8eXmDRK3NJj80
Vkn+uhrk+SHmbjIhmS57Pv9p8TWyRvemph/nMUuZGKBUu2X+JQxggck0KigIrXjsmciCsM
BIM3mYDDfjYbyVhTAAABAQDkV8O1bWUsAIqk7RU+iDZojN5kaO+zUvj1TafX8QX1sY6pu4
Z2cfSEka1532BaehM95bQm7BCPw4cYg56XidmCQTZ9WaWqxVrOo48EKXUtZMZx6nKFOKlq
MT2XTMnGT9n7kFCfEjSVkAjuJ9ZTFLOaoXAaVRnxeHQwOKaup5KKP9GSzNIw328U+96s3V
WKHeT4pMjHBccgW/qX/tRRidZw5in5uBC9Ew5y3UACFTkNOnhUwVfyUNbBZJ2W36msQ3KD
AN7nOrQHqhd3NFyCEy2ovIAKVBacr/VEX6EsRUshIehJzz8EY9f3kXL7WT2QDoz2giPeBJ
HJdEpXt43UpszjAAABAQDVNpqNdHUlCs9XnbIvc6ZRrNh79wt65YFfvh/QEuA33KnA6Ri6
EgnV5IdUWXS/UFaYcm2udydrBpVIVifSYl3sioHBylpri23BEy38PKwVXvghUtfpN6dWGn
NZUG25fQPtIzqi+lo953ZjIj+Adi17AeVv4P4NiLrZeM9lXfWf2pEPOecxXs1IwAf9IiDQ
WepAwRLsu42eEnHA+DSJPZUkSbISfM5X345k0g6EVATX/yLL3CsqClPzPtsqjh6rbEfFg3
2OfIMcWV77gOlGWGQ+bUHc8kV6xJqV9QVacLWzfLvIqHF0wQMf8WLOVHEzkfiq4VjwhVqr
/+FFvljm5nSDAAAAEW1pa2VAQzAyWTUwVEdKR0g4AQ==
-----END OPENSSH PRIVATE KEY-----
+1
View File
@@ -0,0 +1 @@
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC+Lbo0NDxxhNPO2o971mAXkdH+sijkWgOAvk1ttkpbqHS3E/0V3yyJ30GCaj5oaR4zfbu09VUxDpQWZ1GSw4IgNZAOM0sXxvpL4W73m28wpJsat5GhHfY6+BkpUjoqY6wXjPgYi/2do+mVmzC0oRSFJIj7+A90Cz3B/BqWpnvt7zZAnvbttgx4LeXO6hP4oEh1i58ng5CEu2keuqEHmyfuzr6gK/+7zkZKWflIgLXweLa32Vbi2Bvm0fw/SQQMNYZ89LmiSoaLiQVO/AFc4JLavh2GLmFGSk/s+obienQ4J+8rkhFEnbt8KePSWhz6m5kkDGc8xu4mvri/DFHHvw8z9XQ8NSyXQ/ppH+9t7ReYX0UaptZ/ZNqvWnivoL39szNwPPyx5hPyHMsgc/f5GXWF2WxMub7UTOFoTEoh362Nd6AoYbidTbiHqocSdHEtAMzASx2R+PgTWlXKlkldXI2P30Ojzi+LQRSe7tHgeZBv8L4+7uowO0tPdmTs7kyDgtD5xPP1XSehgQDIbjqcAHhT+cnMGdXUn5oB2mKgeZQBiEtiVD0qLLssImL3aD1aprjDEwrEFs7ll4f76nanaVSL+hA2rH/Z0tgWbLTDG8kdtkiEBoVs54K8eDW5wgBDOeWLaM6vaqgH7IpFE8Cn/c1edmDW/tx7YKTCIAPnbCO0KQ== mike@C02Y50TGJGH8
@@ -0,0 +1,11 @@
Host *
AddressFamily inet
Host {{hostname}}
ProxyCommand bash -c '/usr/local/bin/cloudflared access ssh-gen --hostname %h; ssh -F /root/.ssh/short_lived_cert_config -tt %r@cfpipe-{{hostname}} >&2 <&1'
Host cfpipe-{{hostname}}
HostName {{hostname}}
ProxyCommand /usr/local/bin/cloudflared access ssh --hostname %h
IdentityFile ~/.cloudflared/{{hostname}}-cf_key
CertificateFile ~/.cloudflared/{{hostname}}-cf_key-cert.pub
+195
View File
@@ -0,0 +1,195 @@
"""
Cloudflared Integration tests
"""
import unittest
import subprocess
import os
import tempfile
from contextlib import contextmanager
from pexpect import pxssh
class TestSSHBase(unittest.TestCase):
"""
SSH test base class containing constants and helper funcs
"""
HOSTNAME = os.environ["SSH_HOSTNAME"]
SSH_USER = os.environ["SSH_USER"]
SSH_TARGET = f"{SSH_USER}@{HOSTNAME}"
AUTHORIZED_KEYS_SSH_CONFIG = os.environ["AUTHORIZED_KEYS_SSH_CONFIG"]
SHORT_LIVED_CERT_SSH_CONFIG = os.environ["SHORT_LIVED_CERT_SSH_CONFIG"]
SSH_OPTIONS = {"StrictHostKeyChecking": "no"}
@classmethod
def get_ssh_command(cls, pty=True):
"""
Return ssh command arg list. If pty is true, a PTY is forced for the session.
"""
cmd = [
"ssh",
"-o",
"StrictHostKeyChecking=no",
"-F",
cls.AUTHORIZED_KEYS_SSH_CONFIG,
cls.SSH_TARGET,
]
if not pty:
cmd += ["-T"]
else:
cmd += ["-tt"]
return cmd
@classmethod
@contextmanager
def ssh_session_manager(cls, *args, **kwargs):
"""
Context manager for interacting with a pxssh session.
Disables pty echo on the remote server and ensures session is terminated afterward.
"""
session = pxssh.pxssh(options=cls.SSH_OPTIONS)
session.login(
cls.HOSTNAME,
username=cls.SSH_USER,
original_prompt=r"[#@$]",
ssh_config=kwargs.get("ssh_config", cls.AUTHORIZED_KEYS_SSH_CONFIG),
ssh_tunnels=kwargs.get("ssh_tunnels", {}),
)
try:
session.sendline("stty -echo")
session.prompt()
yield session
finally:
session.logout()
@staticmethod
def get_command_output(session, cmd):
"""
Executes command on remote ssh server and waits for prompt.
Returns command output
"""
session.sendline(cmd)
session.prompt()
return session.before.decode().strip()
def exec_command(self, cmd, shell=False):
"""
Executes command locally. Raises Assertion error for non-zero return code.
Returns stdout and stderr
"""
proc = subprocess.Popen(
cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE, shell=shell
)
raw_out, raw_err = proc.communicate()
out = raw_out.decode()
err = raw_err.decode()
self.assertEqual(proc.returncode, 0, msg=f"stdout: {out} stderr: {err}")
return out.strip(), err.strip()
class TestSSHCommandExec(TestSSHBase):
"""
Tests inline ssh command exec
"""
# Name of file to be downloaded over SCP on remote server.
REMOTE_SCP_FILENAME = os.environ["REMOTE_SCP_FILENAME"]
@classmethod
def get_scp_base_command(cls):
return [
"scp",
"-o",
"StrictHostKeyChecking=no",
"-v",
"-F",
cls.AUTHORIZED_KEYS_SSH_CONFIG,
]
@unittest.skip(
"This creates files on the remote. Should be skipped until server is dockerized."
)
def test_verbose_scp_sink_mode(self):
with tempfile.NamedTemporaryFile() as fl:
self.exec_command(
self.get_scp_base_command() + [fl.name, f"{self.SSH_TARGET}:"]
)
def test_verbose_scp_source_mode(self):
with tempfile.TemporaryDirectory() as tmpdirname:
self.exec_command(
self.get_scp_base_command()
+ [f"{self.SSH_TARGET}:{self.REMOTE_SCP_FILENAME}", tmpdirname]
)
local_filename = os.path.join(tmpdirname, self.REMOTE_SCP_FILENAME)
self.assertTrue(os.path.exists(local_filename))
self.assertTrue(os.path.getsize(local_filename) > 0)
def test_pty_command(self):
base_cmd = self.get_ssh_command()
out, _ = self.exec_command(base_cmd + ["whoami"])
self.assertEqual(out.strip().lower(), self.SSH_USER.lower())
out, _ = self.exec_command(base_cmd + ["tty"])
self.assertNotEqual(out, "not a tty")
def test_non_pty_command(self):
base_cmd = self.get_ssh_command(pty=False)
out, _ = self.exec_command(base_cmd + ["whoami"])
self.assertEqual(out.strip().lower(), self.SSH_USER.lower())
out, _ = self.exec_command(base_cmd + ["tty"])
self.assertEqual(out, "not a tty")
class TestSSHShell(TestSSHBase):
"""
Tests interactive SSH shell
"""
# File path to a file on the remote server with root only read privileges.
ROOT_ONLY_TEST_FILE_PATH = os.environ["ROOT_ONLY_TEST_FILE_PATH"]
def test_ssh_pty(self):
with self.ssh_session_manager() as session:
# Test shell launched as correct user
username = self.get_command_output(session, "whoami")
self.assertEqual(username.lower(), self.SSH_USER.lower())
# Test USER env variable set
user_var = self.get_command_output(session, "echo $USER")
self.assertEqual(user_var.lower(), self.SSH_USER.lower())
# Test HOME env variable set to true user home.
home_env = self.get_command_output(session, "echo $HOME")
pwd = self.get_command_output(session, "pwd")
self.assertEqual(pwd, home_env)
# Test shell launched in correct user home dir.
self.assertIn(username, pwd)
# Ensure shell launched with correct user's permissions and privs.
# Cant read root owned 0700 files.
output = self.get_command_output(
session, f"cat {self.ROOT_ONLY_TEST_FILE_PATH}"
)
self.assertIn("Permission denied", output)
def test_short_lived_cert_auth(self):
with self.ssh_session_manager(
ssh_config=self.SHORT_LIVED_CERT_SSH_CONFIG
) as session:
username = self.get_command_output(session, "whoami")
self.assertEqual(username.lower(), self.SSH_USER.lower())
unittest.main()
+22 -16
View File
@@ -8,7 +8,6 @@ import (
"crypto/x509"
"encoding/json"
"encoding/pem"
"errors"
"fmt"
"io"
"io/ioutil"
@@ -20,6 +19,7 @@ import (
cfpath "github.com/cloudflare/cloudflared/cmd/cloudflared/path"
"github.com/coreos/go-oidc/jose"
homedir "github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
gossh "golang.org/x/crypto/ssh"
)
@@ -73,48 +73,54 @@ func GenerateShortLivedCertificate(appURL *url.URL, token string) error {
// handleCertificateGeneration takes a JWT and uses it build a signPayload
// to send to the Sign endpoint with the public key from the keypair it generated
func handleCertificateGeneration(token, fullName string) (string, error) {
pub, err := generateKeyPair(fullName)
if err != nil {
return "", err
}
return SignCert(token, string(pub))
}
func SignCert(token, pubKey string) (string, error) {
if token == "" {
return "", errors.New("invalid token")
}
jwt, err := jose.ParseJWT(token)
if err != nil {
return "", err
return "", errors.Wrap(err, "failed to parse JWT")
}
claims, err := jwt.Claims()
if err != nil {
return "", err
return "", errors.Wrap(err, "failed to retrieve JWT claims")
}
issuer, _, err := claims.StringClaim("iss")
if err != nil {
return "", err
}
pub, err := generateKeyPair(fullName)
if err != nil {
return "", err
return "", errors.Wrap(err, "failed to retrieve JWT iss")
}
buf, err := json.Marshal(&signPayload{
PublicKey: string(pub),
PublicKey: pubKey,
JWT: token,
Issuer: issuer,
})
if err != nil {
return "", err
return "", errors.Wrap(err, "failed to marshal signPayload")
}
var res *http.Response
if mockRequest != nil {
res, err = mockRequest(issuer+signEndpoint, "application/json", bytes.NewBuffer(buf))
} else {
res, err = http.Post(issuer+signEndpoint, "application/json", bytes.NewBuffer(buf))
client := http.Client{
Timeout: 10 * time.Second,
}
res, err = client.Post(issuer+signEndpoint, "application/json", bytes.NewBuffer(buf))
}
if err != nil {
return "", err
return "", errors.Wrap(err, "failed to send request")
}
defer res.Body.Close()
@@ -130,9 +136,9 @@ func handleCertificateGeneration(token, fullName string) (string, error) {
var signRes signResponse
if err := decoder.Decode(&signRes); err != nil {
return "", err
return "", errors.Wrap(err, "failed to decode HTTP response")
}
return signRes.Certificate, err
return signRes.Certificate, nil
}
// generateKeyPair creates a EC keypair (P256) and stores them in the homedir.
+37
View File
@@ -0,0 +1,37 @@
package sshlog
import (
"io"
"github.com/sirupsen/logrus"
)
//empty manager implements the Manager but does nothing (for testing and to disable logging unless the logs are set)
type emptyManager struct {
}
type emptyWriteCloser struct {
}
// NewEmptyManager creates a new instance of a log empty log manager that does nothing
func NewEmptyManager() Manager {
return &emptyManager{}
}
func (m *emptyManager) NewLogger(name string, logger *logrus.Logger) (io.WriteCloser, error) {
return &emptyWriteCloser{}, nil
}
func (m *emptyManager) NewSessionLogger(name string, logger *logrus.Logger) (io.WriteCloser, error) {
return &emptyWriteCloser{}, nil
}
// emptyWriteCloser
func (w *emptyWriteCloser) Write(p []byte) (n int, err error) {
return len(p), nil
}
func (w *emptyWriteCloser) Close() error {
return nil
}
+15
View File
@@ -0,0 +1,15 @@
# Generate go.capnp.out with:
# capnp compile -o- go.capnp > go.capnp.out
# Must run inside this directory to preserve paths.
@0xd12a1c51fedd6c88;
annotation package(file) :Text;
annotation import(file) :Text;
annotation doc(struct, field, enum) :Text;
annotation tag(enumerant) :Text;
annotation notag(enumerant) :Void;
annotation customtype(field) :Text;
annotation name(struct, field, union, enum, enumerant, interface, method, param, annotation, const, group) :Text;
$package("capnp");
+26 -15
View File
@@ -2,6 +2,7 @@ package sshlog
import (
"bufio"
"errors"
"fmt"
"os"
"path/filepath"
@@ -12,19 +13,22 @@ import (
)
const (
logTimeFormat = "2006-01-02T15-04-05.000"
megabyte = 1024 * 1024
logTimeFormat = "2006-01-02T15-04-05.000"
megabyte = 1024 * 1024
defaultFileSizeLimit = 100 * megabyte
)
// Logger will buffer and write events to disk
type Logger struct {
sync.Mutex
filename string
file *os.File
writeBuffer *bufio.Writer
logger *logrus.Logger
done chan struct{}
once sync.Once
filename string
file *os.File
writeBuffer *bufio.Writer
logger *logrus.Logger
flushInterval time.Duration
maxFileSize int64
done chan struct{}
once sync.Once
}
// NewLogger creates a Logger instance. A buffer is created that needs to be
@@ -34,16 +38,23 @@ type Logger struct {
// logger variable is a logrus that will log all i/o, filesystem error etc, that
// that shouldn't end execution of the logger, but are useful to report to the
// caller.
func NewLogger(filename string, logger *logrus.Logger) (*Logger, error) {
func NewLogger(filename string, logger *logrus.Logger, flushInterval time.Duration, maxFileSize int64) (*Logger, error) {
if logger == nil {
return nil, errors.New("logger can't be nil")
}
f, err := os.OpenFile(filename, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, os.FileMode(0600))
if err != nil {
return nil, err
}
l := &Logger{filename: filename,
file: f,
writeBuffer: bufio.NewWriter(f),
logger: logger,
done: make(chan struct{})}
file: f,
writeBuffer: bufio.NewWriter(f),
logger: logger,
flushInterval: flushInterval,
maxFileSize: maxFileSize,
done: make(chan struct{}),
}
go l.writer()
return l, nil
}
@@ -70,7 +81,7 @@ func (l *Logger) Close() error {
// writer is the run loop that handles draining the write buffer and syncing
// data to disk.
func (l *Logger) writer() {
ticker := time.NewTicker(time.Second)
ticker := time.NewTicker(l.flushInterval)
defer ticker.Stop()
for {
select {
@@ -118,7 +129,7 @@ func (l *Logger) shouldRotate() bool {
return false
}
return info.Size() >= 100*megabyte
return info.Size() >= l.maxFileSize
}
// rotate creates a new logfile with the existing filename and renames the
+89
View File
@@ -0,0 +1,89 @@
package sshlog
import (
"log"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/sirupsen/logrus"
)
const logFileName = "test-logger.log"
func createLogger(t *testing.T) *Logger {
os.Remove(logFileName)
l := logrus.New()
logger, err := NewLogger(logFileName, l, time.Millisecond, 1024)
if err != nil {
t.Fatal("couldn't create the logger!", err)
}
return logger
}
// AUTH-2115 TODO: fix this test
//func TestWrite(t *testing.T) {
// testStr := "hi"
// logger := createLogger(t)
// defer func() {
// logger.Close()
// os.Remove(logFileName)
// }()
//
// logger.Write([]byte(testStr))
// time.Sleep(2 * time.Millisecond)
// data, err := ioutil.ReadFile(logFileName)
// if err != nil {
// t.Fatal("couldn't read the log file!", err)
// }
// checkStr := string(data)
// if checkStr != testStr {
// t.Fatal("file data doesn't match!")
// }
//}
func TestFilenameRotation(t *testing.T) {
newName := rotationName("dir/bob/acoolloggername.log")
dir := filepath.Dir(newName)
if dir != "dir/bob" {
t.Fatal("rotation name doesn't respect the directory filepath:", newName)
}
filename := filepath.Base(newName)
if !strings.HasPrefix(filename, "acoolloggername") {
t.Fatal("rotation filename is wrong:", filename)
}
ext := filepath.Ext(newName)
if ext != ".log" {
t.Fatal("rotation file extension is wrong:", ext)
}
}
func TestRotation(t *testing.T) {
logger := createLogger(t)
for i := 0; i < 2000; i++ {
logger.Write([]byte("a string for testing rotation\n"))
}
logger.Close()
count := 0
filepath.Walk(".", func(path string, info os.FileInfo, err error) error {
if err != nil || info.IsDir() {
return nil
}
if strings.HasPrefix(info.Name(), "test-logger") {
log.Println("deleting: ", path)
os.Remove(path)
count++
}
return nil
})
if count < 2 {
t.Fatal("rotation didn't roll files:", count)
}
}
+15 -4
View File
@@ -2,6 +2,8 @@ package sshlog
import (
"io"
"path/filepath"
"time"
"github.com/sirupsen/logrus"
)
@@ -9,15 +11,24 @@ import (
// Manager be managing logs bruh
type Manager interface {
NewLogger(string, *logrus.Logger) (io.WriteCloser, error)
NewSessionLogger(string, *logrus.Logger) (io.WriteCloser, error)
}
type manager struct{}
type manager struct {
baseDirectory string
}
// New creates a new instance of a log manager
func New() Manager {
return &manager{}
func New(baseDirectory string) Manager {
return &manager{
baseDirectory: baseDirectory,
}
}
func (m *manager) NewLogger(name string, logger *logrus.Logger) (io.WriteCloser, error) {
return NewLogger(name, logger)
return NewLogger(filepath.Join(m.baseDirectory, name), logger, time.Second, defaultFileSizeLimit)
}
func (m *manager) NewSessionLogger(name string, logger *logrus.Logger) (io.WriteCloser, error) {
return NewSessionLogger(filepath.Join(m.baseDirectory, name), logger, time.Second, defaultFileSizeLimit)
}
+9
View File
@@ -0,0 +1,9 @@
using Go = import "go.capnp";
@0x8f43375162194466;
$Go.package("sshlog");
$Go.import("github.com/cloudflare/cloudflared/sshlog");
struct SessionLog {
timestamp @0 :Text;
content @1 :Data;
}
+110
View File
@@ -0,0 +1,110 @@
// Code generated by capnpc-go. DO NOT EDIT.
package sshlog
import (
capnp "zombiezen.com/go/capnproto2"
text "zombiezen.com/go/capnproto2/encoding/text"
schemas "zombiezen.com/go/capnproto2/schemas"
)
type SessionLog struct{ capnp.Struct }
// SessionLog_TypeID is the unique identifier for the type SessionLog.
const SessionLog_TypeID = 0xa13a07c504a5ab64
func NewSessionLog(s *capnp.Segment) (SessionLog, error) {
st, err := capnp.NewStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2})
return SessionLog{st}, err
}
func NewRootSessionLog(s *capnp.Segment) (SessionLog, error) {
st, err := capnp.NewRootStruct(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2})
return SessionLog{st}, err
}
func ReadRootSessionLog(msg *capnp.Message) (SessionLog, error) {
root, err := msg.RootPtr()
return SessionLog{root.Struct()}, err
}
func (s SessionLog) String() string {
str, _ := text.Marshal(0xa13a07c504a5ab64, s.Struct)
return str
}
func (s SessionLog) Timestamp() (string, error) {
p, err := s.Struct.Ptr(0)
return p.Text(), err
}
func (s SessionLog) HasTimestamp() bool {
p, err := s.Struct.Ptr(0)
return p.IsValid() || err != nil
}
func (s SessionLog) TimestampBytes() ([]byte, error) {
p, err := s.Struct.Ptr(0)
return p.TextBytes(), err
}
func (s SessionLog) SetTimestamp(v string) error {
return s.Struct.SetText(0, v)
}
func (s SessionLog) Content() ([]byte, error) {
p, err := s.Struct.Ptr(1)
return []byte(p.Data()), err
}
func (s SessionLog) HasContent() bool {
p, err := s.Struct.Ptr(1)
return p.IsValid() || err != nil
}
func (s SessionLog) SetContent(v []byte) error {
return s.Struct.SetData(1, v)
}
// SessionLog_List is a list of SessionLog.
type SessionLog_List struct{ capnp.List }
// NewSessionLog creates a new list of SessionLog.
func NewSessionLog_List(s *capnp.Segment, sz int32) (SessionLog_List, error) {
l, err := capnp.NewCompositeList(s, capnp.ObjectSize{DataSize: 0, PointerCount: 2}, sz)
return SessionLog_List{l}, err
}
func (s SessionLog_List) At(i int) SessionLog { return SessionLog{s.List.Struct(i)} }
func (s SessionLog_List) Set(i int, v SessionLog) error { return s.List.SetStruct(i, v.Struct) }
func (s SessionLog_List) String() string {
str, _ := text.MarshalList(0xa13a07c504a5ab64, s.List)
return str
}
// SessionLog_Promise is a wrapper for a SessionLog promised by a client call.
type SessionLog_Promise struct{ *capnp.Pipeline }
func (p SessionLog_Promise) Struct() (SessionLog, error) {
s, err := p.Pipeline.Struct()
return SessionLog{s}, err
}
const schema_8f43375162194466 = "x\xda\x120q`\x12d\x8dg`\x08dae\xfb" +
"\x9f\xb2z)\xcbQv\xab\x85\x0c\x82B\x8c\xff\xd3\\" +
"$\x93\x02\xcd\x9d\xfb\x19X\x99\xd8\x19\x18\x04E_\x09" +
"*\x82h\xd9r\x06\xc6\xff\xc5\xa9\xc5\xc5\x99\xf9y\xf1" +
"L9\xf9\xe9z\xc9\x89\x05y\x05V\xc1`!\xfe<" +
"\x9f\xfc\xf4\x00F\xc6@\x0ef\x16\x06\x06\x16F\x06\x06" +
"A\xcd \x06\x86@\x0df\xc6@\x13&FAFF" +
"\x11F\x90\xa0\xa1\x13\x03C\xa0\x0e3c\xa0\x05\x13\xe3" +
"\xff\x92\xcc\xdc\xd4\xe2\x92\xc4\\\x06\xc6\x02F\x1e\x06&" +
"F\x1e\x06\xc6\xfa\xe4\xfc\xbc\x92\xd4\xbc\x12F^\x06&" +
"F^\x06F@\x00\x00\x00\xff\xff\xdaK$\x1a"
func init() {
schemas.Register(schema_8f43375162194466,
0xa13a07c504a5ab64)
}
+71
View File
@@ -0,0 +1,71 @@
package sshlog
import (
"time"
"github.com/sirupsen/logrus"
capnp "zombiezen.com/go/capnproto2"
"zombiezen.com/go/capnproto2/pogs"
)
// SessionLogger will buffer and write events to disk using capnp proto for session replay
type SessionLogger struct {
logger *Logger
encoder *capnp.Encoder
}
type sessionLogData struct {
Timestamp string // The UTC timestamp of when the log occurred
Content []byte // The shell output
}
// NewSessionLogger creates a new session logger by encapsulating a Logger object and writing capnp encoded messages to it
func NewSessionLogger(filename string, logger *logrus.Logger, flushInterval time.Duration, maxFileSize int64) (*SessionLogger, error) {
l, err := NewLogger(filename, logger, flushInterval, maxFileSize)
if err != nil {
return nil, err
}
sessionLogger := &SessionLogger{
logger: l,
encoder: capnp.NewEncoder(l),
}
return sessionLogger, nil
}
// Writes to a log buffer. Implements the io.Writer interface.
func (l *SessionLogger) Write(p []byte) (n int, err error) {
return l.writeSessionLog(&sessionLogData{
Timestamp: time.Now().UTC().Format(time.RFC3339),
Content: p,
})
}
// Close drains anything left in the buffer and cleans up any resources still
// in use.
func (l *SessionLogger) Close() error {
return l.logger.Close()
}
func (l *SessionLogger) writeSessionLog(p *sessionLogData) (int, error) {
msg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))
if err != nil {
return 0, err
}
log, err := NewRootSessionLog(seg)
if err != nil {
return 0, err
}
log.SetTimestamp(p.Timestamp)
log.SetContent(p.Content)
if err := l.encoder.Encode(msg); err != nil {
return 0, err
}
return len(p.Content), nil
}
func unmarshalSessionLog(s SessionLog) (*sessionLogData, error) {
p := new(sessionLogData)
err := pogs.Extract(p, SessionLog_TypeID, s.Struct)
return p, err
}
+69
View File
@@ -0,0 +1,69 @@
package sshlog
import (
"os"
"testing"
"time"
"github.com/sirupsen/logrus"
capnp "zombiezen.com/go/capnproto2"
)
const sessionLogFileName = "test-session-logger.log"
func createSessionLogger(t *testing.T) *SessionLogger {
os.Remove(sessionLogFileName)
l := logrus.New()
logger, err := NewSessionLogger(sessionLogFileName, l, time.Millisecond, 1024)
if err != nil {
t.Fatal("couldn't create the logger!", err)
}
return logger
}
func TestSessionLogWrite(t *testing.T) {
testStr := "hi"
logger := createSessionLogger(t)
defer func() {
logger.Close()
os.Remove(sessionLogFileName)
}()
logger.Write([]byte(testStr))
time.Sleep(2 * time.Millisecond)
f, err := os.Open(sessionLogFileName)
if err != nil {
t.Fatal("couldn't read the log file!", err)
}
defer f.Close()
msg, err := capnp.NewDecoder(f).Decode()
if err != nil {
t.Fatal("couldn't read the capnp msg file!", err)
}
sessionLog, err := ReadRootSessionLog(msg)
if err != nil {
t.Fatal("couldn't read the session log from the msg!", err)
}
timeStr, err := sessionLog.Timestamp()
if err != nil {
t.Fatal("couldn't read the Timestamp field!", err)
}
_, terr := time.Parse(time.RFC3339, timeStr)
if terr != nil {
t.Fatal("couldn't parse the Timestamp into the expected RFC3339 format", terr)
}
data, err := sessionLog.Content()
if err != nil {
t.Fatal("couldn't read the Content field!", err)
}
checkStr := string(data)
if checkStr != testStr {
t.Fatal("file data doesn't match!")
}
}
-109
View File
@@ -1,109 +0,0 @@
//+build !windows
package sshserver
import (
"fmt"
"io/ioutil"
"os"
"path"
"github.com/gliderlabs/ssh"
"github.com/pkg/errors"
gossh "golang.org/x/crypto/ssh"
)
var (
systemConfigPath = "/etc/cloudflared/"
authorizedKeysDir = ".cloudflared/authorized_keys"
)
func (s *SSHServer) configureAuthentication() {
caCert, err := getCACert()
if err != nil {
s.logger.Info(err)
}
s.caCert = caCert
s.PublicKeyHandler = s.authenticationHandler
}
func (s *SSHServer) authenticationHandler(ctx ssh.Context, key ssh.PublicKey) bool {
cert, ok := key.(*gossh.Certificate)
if !ok {
return s.authorizedKeyHandler(ctx, key)
}
return s.shortLivedCertHandler(ctx, cert)
}
func (s *SSHServer) authorizedKeyHandler(ctx ssh.Context, key ssh.PublicKey) bool {
sshUser, err := s.getUserFunc(ctx.User())
if err != nil {
s.logger.Debugf("Invalid user: %s", ctx.User())
return false
}
authorizedKeysPath := path.Join(sshUser.HomeDir, authorizedKeysDir)
if _, err := os.Stat(authorizedKeysPath); os.IsNotExist(err) {
s.logger.Debugf("authorized_keys file %s not found", authorizedKeysPath)
return false
}
authorizedKeysBytes, err := ioutil.ReadFile(authorizedKeysPath)
if err != nil {
s.logger.WithError(err).Errorf("Failed to load authorized_keys %s", authorizedKeysPath)
return false
}
for len(authorizedKeysBytes) > 0 {
// Skips invalid keys. Returns error if no valid keys remain.
pubKey, _, _, rest, err := ssh.ParseAuthorizedKey(authorizedKeysBytes)
authorizedKeysBytes = rest
if err != nil {
s.logger.Errorf("Invalid key(s) found in %s", authorizedKeysPath)
return false
}
if ssh.KeysEqual(pubKey, key) {
ctx.SetValue("sshUser", sshUser)
return true
}
}
s.logger.Debugf("Matching public key not found in %s", authorizedKeysPath)
return false
}
func (s *SSHServer) shortLivedCertHandler(ctx ssh.Context, cert *gossh.Certificate) bool {
if !ssh.KeysEqual(s.caCert, cert.SignatureKey) {
s.logger.Debug("CA certificate does not match user certificate signer")
return false
}
checker := gossh.CertChecker{}
if err := checker.CheckCert(ctx.User(), cert); err != nil {
s.logger.Debug(err)
return false
} else {
sshUser, err := s.getUserFunc(ctx.User())
if err != nil {
s.logger.Debugf("Invalid user: %s", ctx.User())
return false
}
ctx.SetValue("sshUser", sshUser)
}
return true
}
func getCACert() (ssh.PublicKey, error) {
caCertPath := path.Join(systemConfigPath, "ca.pub")
caCertBytes, err := ioutil.ReadFile(caCertPath)
if err != nil {
return nil, errors.Wrap(err, fmt.Sprintf("Failed to load CA certertificate %s", caCertPath))
}
caCert, _, _, _, err := ssh.ParseAuthorizedKey(caCertBytes)
if err != nil {
return nil, errors.Wrap(err, "Failed to parse CA Certificate")
}
return caCert, nil
}
-197
View File
@@ -1,197 +0,0 @@
//+build !windows
package sshserver
import (
"context"
"io/ioutil"
"net"
"os"
"os/user"
"path"
"sync"
"testing"
"github.com/cloudflare/cloudflared/log"
"github.com/gliderlabs/ssh"
"github.com/sirupsen/logrus"
"github.com/sirupsen/logrus/hooks/test"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
gossh "golang.org/x/crypto/ssh"
)
const (
validPrincipal = "testUser"
testDir = "testdata"
testUserKeyFilename = "id_rsa.pub"
testCAFilename = "ca.pub"
testOtherCAFilename = "other_ca.pub"
testUserCertFilename = "id_rsa-cert.pub"
)
var logger, hook = test.NewNullLogger()
func TestMain(m *testing.M) {
authorizedKeysDir = testUserKeyFilename
logger.SetLevel(logrus.DebugLevel)
code := m.Run()
os.Exit(code)
}
func TestPublicKeyAuth_Success(t *testing.T) {
context, cancel := newMockContext(validPrincipal)
defer cancel()
sshServer := SSHServer{getUserFunc: getMockUser}
pubKey := getKey(t, testUserKeyFilename)
assert.True(t, sshServer.authorizedKeyHandler(context, pubKey))
}
func TestPublicKeyAuth_MissingKey(t *testing.T) {
context, cancel := newMockContext(validPrincipal)
defer cancel()
sshServer := SSHServer{logger: logger, getUserFunc: getMockUser}
pubKey := getKey(t, testOtherCAFilename)
assert.False(t, sshServer.authorizedKeyHandler(context, pubKey))
assert.Contains(t, hook.LastEntry().Message, "Matching public key not found in")
}
func TestPublicKeyAuth_InvalidUser(t *testing.T) {
context, cancel := newMockContext("notAUser")
defer cancel()
sshServer := SSHServer{logger: logger, getUserFunc: lookupUser}
pubKey := getKey(t, testUserKeyFilename)
assert.False(t, sshServer.authorizedKeyHandler(context, pubKey))
assert.Contains(t, hook.LastEntry().Message, "Invalid user")
}
func TestPublicKeyAuth_MissingFile(t *testing.T) {
currentUser, err := user.Current()
require.Nil(t, err)
context, cancel := newMockContext(currentUser.Username)
defer cancel()
sshServer := SSHServer{Server: ssh.Server{}, logger: logger, getUserFunc: lookupUser}
pubKey := getKey(t, testUserKeyFilename)
assert.False(t, sshServer.authorizedKeyHandler(context, pubKey))
assert.Contains(t, hook.LastEntry().Message, "not found")
}
func TestShortLivedCerts_Success(t *testing.T) {
context, cancel := newMockContext(validPrincipal)
defer cancel()
caCert := getKey(t, testCAFilename)
sshServer := SSHServer{logger: log.CreateLogger(), caCert: caCert, getUserFunc: getMockUser}
userCert, ok := getKey(t, testUserCertFilename).(*gossh.Certificate)
require.True(t, ok)
assert.True(t, sshServer.shortLivedCertHandler(context, userCert))
}
func TestShortLivedCerts_CAsDontMatch(t *testing.T) {
context, cancel := newMockContext(validPrincipal)
defer cancel()
caCert := getKey(t, testOtherCAFilename)
sshServer := SSHServer{logger: logger, caCert: caCert, getUserFunc: getMockUser}
userCert, ok := getKey(t, testUserCertFilename).(*gossh.Certificate)
require.True(t, ok)
assert.False(t, sshServer.shortLivedCertHandler(context, userCert))
assert.Equal(t, "CA certificate does not match user certificate signer", hook.LastEntry().Message)
}
func TestShortLivedCerts_UserDoesNotExist(t *testing.T) {
context, cancel := newMockContext(validPrincipal)
defer cancel()
caCert := getKey(t, testCAFilename)
sshServer := SSHServer{logger: logger, caCert: caCert, getUserFunc: lookupUser}
userCert, ok := getKey(t, testUserCertFilename).(*gossh.Certificate)
require.True(t, ok)
assert.False(t, sshServer.shortLivedCertHandler(context, userCert))
assert.Contains(t, hook.LastEntry().Message, "Invalid user")
}
func TestShortLivedCerts_InvalidPrincipal(t *testing.T) {
context, cancel := newMockContext("notAUser")
defer cancel()
caCert := getKey(t, testCAFilename)
sshServer := SSHServer{logger: logger, caCert: caCert, getUserFunc: lookupUser}
userCert, ok := getKey(t, testUserCertFilename).(*gossh.Certificate)
require.True(t, ok)
assert.False(t, sshServer.shortLivedCertHandler(context, userCert))
assert.Contains(t, hook.LastEntry().Message, "not in the set of valid principals for given certificate")
}
func getMockUser(_ string) (*User, error) {
return &User{
Username: validPrincipal,
HomeDir: testDir,
}, nil
}
func getKey(t *testing.T, filename string) ssh.PublicKey {
path := path.Join(testDir, filename)
bytes, err := ioutil.ReadFile(path)
require.Nil(t, err)
pubKey, _, _, _, err := ssh.ParseAuthorizedKey(bytes)
require.Nil(t, err)
return pubKey
}
type mockSSHContext struct {
context.Context
*sync.Mutex
}
func newMockContext(user string) (*mockSSHContext, context.CancelFunc) {
innerCtx, cancel := context.WithCancel(context.Background())
mockCtx := &mockSSHContext{innerCtx, &sync.Mutex{}}
mockCtx.SetValue("user", user)
return mockCtx, cancel
}
func (ctx *mockSSHContext) SetValue(key, value interface{}) {
ctx.Context = context.WithValue(ctx.Context, key, value)
}
func (ctx *mockSSHContext) User() string {
return ctx.Value("user").(string)
}
func (ctx *mockSSHContext) SessionID() string {
return ""
}
func (ctx *mockSSHContext) ClientVersion() string {
return ""
}
func (ctx *mockSSHContext) ServerVersion() string {
return ""
}
func (ctx *mockSSHContext) RemoteAddr() net.Addr {
return nil
}
func (ctx *mockSSHContext) LocalAddr() net.Addr {
return nil
}
func (ctx *mockSSHContext) Permissions() *ssh.Permissions {
return nil
}
-226
View File
@@ -1,226 +0,0 @@
// Taken from https://github.com/golang/go/blob/ad644d2e86bab85787879d41c2d2aebbd7c57db8/src/os/user/user.go
// and modified to return login shell in User struct. cloudflared requires cgo for compilation because of this addition.
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build aix darwin dragonfly freebsd !android,linux netbsd openbsd solaris
// +build cgo,!osusergo
package sshserver
import (
"fmt"
"strconv"
"strings"
"syscall"
"unsafe"
)
/*
#cgo solaris CFLAGS: -D_POSIX_PTHREAD_SEMANTICS
#include <unistd.h>
#include <sys/types.h>
#include <pwd.h>
#include <grp.h>
#include <stdlib.h>
static int mygetpwuid_r(int uid, struct passwd *pwd,
char *buf, size_t buflen, struct passwd **result) {
return getpwuid_r(uid, pwd, buf, buflen, result);
}
static int mygetpwnam_r(const char *name, struct passwd *pwd,
char *buf, size_t buflen, struct passwd **result) {
return getpwnam_r(name, pwd, buf, buflen, result);
}
static int mygetgrgid_r(int gid, struct group *grp,
char *buf, size_t buflen, struct group **result) {
return getgrgid_r(gid, grp, buf, buflen, result);
}
static int mygetgrnam_r(const char *name, struct group *grp,
char *buf, size_t buflen, struct group **result) {
return getgrnam_r(name, grp, buf, buflen, result);
}
*/
import "C"
type UnknownUserIdError int
func (e UnknownUserIdError) Error() string {
return "user: unknown userid " + strconv.Itoa(int(e))
}
// UnknownUserError is returned by Lookup when
// a user cannot be found.
type UnknownUserError string
func (e UnknownUserError) Error() string {
return "user: unknown user " + string(e)
}
// UnknownGroupIdError is returned by LookupGroupId when
// a group cannot be found.
type UnknownGroupIdError string
func (e UnknownGroupIdError) Error() string {
return "group: unknown groupid " + string(e)
}
// UnknownGroupError is returned by LookupGroup when
// a group cannot be found.
type UnknownGroupError string
func (e UnknownGroupError) Error() string {
return "group: unknown group " + string(e)
}
type User struct {
// Uid is the user ID.
// On POSIX systems, this is a decimal number representing the uid.
// On Windows, this is a security identifier (SID) in a string format.
// On Plan 9, this is the contents of /dev/user.
Uid string
// Gid is the primary group ID.
// On POSIX systems, this is a decimal number representing the gid.
// On Windows, this is a SID in a string format.
// On Plan 9, this is the contents of /dev/user.
Gid string
// Username is the login name.
Username string
// Name is the user's real or display name.
// It might be blank.
// On POSIX systems, this is the first (or only) entry in the GECOS field
// list.
// On Windows, this is the user's display name.
// On Plan 9, this is the contents of /dev/user.
Name string
// HomeDir is the path to the user's home directory (if they have one).
HomeDir string
/****************** Begin added code ******************/
// Login shell
Shell string
/****************** End added code ******************/
}
func lookupUser(username string) (*User, error) {
var pwd C.struct_passwd
var result *C.struct_passwd
nameC := make([]byte, len(username)+1)
copy(nameC, username)
buf := alloc(userBuffer)
defer buf.free()
err := retryWithBuffer(buf, func() syscall.Errno {
// mygetpwnam_r is a wrapper around getpwnam_r to avoid
// passing a size_t to getpwnam_r, because for unknown
// reasons passing a size_t to getpwnam_r doesn't work on
// Solaris.
return syscall.Errno(C.mygetpwnam_r((*C.char)(unsafe.Pointer(&nameC[0])),
&pwd,
(*C.char)(buf.ptr),
C.size_t(buf.size),
&result))
})
if err != nil {
return nil, fmt.Errorf("user: lookup username %s: %v", username, err)
}
if result == nil {
return nil, UnknownUserError(username)
}
return buildUser(&pwd), err
}
func buildUser(pwd *C.struct_passwd) *User {
u := &User{
Uid: strconv.FormatUint(uint64(pwd.pw_uid), 10),
Gid: strconv.FormatUint(uint64(pwd.pw_gid), 10),
Username: C.GoString(pwd.pw_name),
Name: C.GoString(pwd.pw_gecos),
HomeDir: C.GoString(pwd.pw_dir),
/****************** Begin added code ******************/
Shell: C.GoString(pwd.pw_shell),
/****************** End added code ******************/
}
// The pw_gecos field isn't quite standardized. Some docs
// say: "It is expected to be a comma separated list of
// personal data where the first item is the full name of the
// user."
if i := strings.Index(u.Name, ","); i >= 0 {
u.Name = u.Name[:i]
}
return u
}
type bufferKind C.int
const (
userBuffer = bufferKind(C._SC_GETPW_R_SIZE_MAX)
)
func (k bufferKind) initialSize() C.size_t {
sz := C.sysconf(C.int(k))
if sz == -1 {
// DragonFly and FreeBSD do not have _SC_GETPW_R_SIZE_MAX.
// Additionally, not all Linux systems have it, either. For
// example, the musl libc returns -1.
return 1024
}
if !isSizeReasonable(int64(sz)) {
// Truncate. If this truly isn't enough, retryWithBuffer will error on the first run.
return maxBufferSize
}
return C.size_t(sz)
}
type memBuffer struct {
ptr unsafe.Pointer
size C.size_t
}
func alloc(kind bufferKind) *memBuffer {
sz := kind.initialSize()
return &memBuffer{
ptr: C.malloc(sz),
size: sz,
}
}
func (mb *memBuffer) resize(newSize C.size_t) {
mb.ptr = C.realloc(mb.ptr, newSize)
mb.size = newSize
}
func (mb *memBuffer) free() {
C.free(mb.ptr)
}
// retryWithBuffer repeatedly calls f(), increasing the size of the
// buffer each time, until f succeeds, fails with a non-ERANGE error,
// or the buffer exceeds a reasonable limit.
func retryWithBuffer(buf *memBuffer, f func() syscall.Errno) error {
for {
errno := f()
if errno == 0 {
return nil
} else if errno != syscall.ERANGE {
return errno
}
newSize := buf.size * 2
if !isSizeReasonable(int64(newSize)) {
return fmt.Errorf("internal buffer exceeds %d bytes", maxBufferSize)
}
buf.resize(newSize)
}
}
const maxBufferSize = 1 << 20
func isSizeReasonable(sz int64) bool {
return sz > 0 && sz <= maxBufferSize
}
+35 -31
View File
@@ -19,46 +19,44 @@ import (
)
const (
rsaFilename = "ssh_host_rsa_key"
ecdsaFilename = "ssh_host_ecdsa_key"
rsaFilename = "ssh_host_rsa_key"
ecdsaFilename = "ssh_host_ecdsa_key"
)
func (s *SSHServer) configureHostKeys() error {
if _, err := os.Stat(systemConfigPath); os.IsNotExist(err) {
if err := os.MkdirAll(systemConfigPath, 0755); err != nil {
return errors.Wrap(err, fmt.Sprintf("Error creating %s directory", systemConfigPath))
var defaultHostKeyDir = filepath.Join(".cloudflared", "host_keys")
func (s *SSHProxy) configureHostKeys(hostKeyDir string) error {
if hostKeyDir == "" {
homeDir, err := os.UserHomeDir()
if err != nil {
return err
}
hostKeyDir = filepath.Join(homeDir, defaultHostKeyDir)
}
if _, err := os.Stat(hostKeyDir); os.IsNotExist(err) {
if err := os.MkdirAll(hostKeyDir, 0755); err != nil {
return errors.Wrap(err, fmt.Sprintf("Error creating %s directory", hostKeyDir))
}
}
if err := s.configureHostKey(s.ensureECDSAKeyExists); err != nil {
if err := s.configureECDSAKey(hostKeyDir); err != nil {
return err
}
if err := s.configureHostKey(s.ensureRSAKeyExists); err != nil {
if err := s.configureRSAKey(hostKeyDir); err != nil {
return err
}
return nil
}
func (s *SSHServer) configureHostKey(keyFunc func() (string, error)) error {
path, err := keyFunc()
if err != nil {
return err
}
if err := s.SetOption(ssh.HostKeyFile(path)); err != nil {
return errors.Wrap(err, "Could not set SSH host key")
}
return nil
}
func (s *SSHServer) ensureRSAKeyExists() (string, error) {
keyPath := filepath.Join(systemConfigPath, rsaFilename)
func (s *SSHProxy) configureRSAKey(basePath string) error {
keyPath := filepath.Join(basePath, rsaFilename)
if _, err := os.Stat(keyPath); os.IsNotExist(err) {
key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
return "", errors.Wrap(err, "Error generating RSA host key")
return errors.Wrap(err, "Error generating RSA host key")
}
privateKey := &pem.Block{
@@ -67,25 +65,28 @@ func (s *SSHServer) ensureRSAKeyExists() (string, error) {
}
if err = writePrivateKey(keyPath, privateKey); err != nil {
return "", err
return err
}
s.logger.Debug("Created new RSA SSH host key: ", keyPath)
}
return keyPath, nil
if err := s.SetOption(ssh.HostKeyFile(keyPath)); err != nil {
return errors.Wrap(err, "Could not set SSH RSA host key")
}
return nil
}
func (s *SSHServer) ensureECDSAKeyExists() (string, error) {
keyPath := filepath.Join(systemConfigPath, ecdsaFilename)
func (s *SSHProxy) configureECDSAKey(basePath string) error {
keyPath := filepath.Join(basePath, ecdsaFilename)
if _, err := os.Stat(keyPath); os.IsNotExist(err) {
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return "", errors.Wrap(err, "Error generating ECDSA host key")
return errors.Wrap(err, "Error generating ECDSA host key")
}
keyBytes, err := x509.MarshalECPrivateKey(key)
if err != nil {
return "", errors.Wrap(err, "Error marshalling ECDSA key")
return errors.Wrap(err, "Error marshalling ECDSA key")
}
privateKey := &pem.Block{
@@ -94,12 +95,15 @@ func (s *SSHServer) ensureECDSAKeyExists() (string, error) {
}
if err = writePrivateKey(keyPath, privateKey); err != nil {
return "", err
return err
}
s.logger.Debug("Created new ECDSA SSH host key: ", keyPath)
}
return keyPath, nil
if err := s.SetOption(ssh.HostKeyFile(keyPath)); err != nil {
return errors.Wrap(err, "Could not set SSH ECDSA host key")
}
return nil
}
func writePrivateKey(keyPath string, privateKey *pem.Block) error {
+29
View File
@@ -0,0 +1,29 @@
package sshserver
import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"testing"
)
func TestHasPort(t *testing.T) {
type testCase struct {
input string
expectedOutput string
}
tests := []testCase{
{"localhost", "localhost:22"},
{"other.addr:22", "other.addr:22"},
{"[2001:db8::1]:8080", "[2001:db8::1]:8080"},
{"[::1]", "[::1]:22"},
{"2001:0db8:3c4d:0015:0000:0000:1a2f:1234", "[2001:0db8:3c4d:0015:0000:0000:1a2f:1234]:22"},
{"::1", "[::1]:22"},
}
for _, test := range tests {
out, err := canonicalizeDest(test.input)
require.Nil(t, err)
assert.Equal(t, test.expectedOutput, out)
}
}
+425 -162
View File
@@ -3,60 +3,110 @@
package sshserver
import (
"errors"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"encoding/binary"
"encoding/json"
"fmt"
"io"
"os"
"os/exec"
"os/user"
"strconv"
"syscall"
"net"
"runtime"
"strings"
"time"
"unsafe"
"github.com/cloudflare/cloudflared/sshgen"
"github.com/cloudflare/cloudflared/sshlog"
"github.com/creack/pty"
"github.com/gliderlabs/ssh"
"github.com/google/uuid"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
gossh "golang.org/x/crypto/ssh"
)
type SSHServer struct {
const (
auditEventStart = "session_start"
auditEventStop = "session_stop"
auditEventExec = "exec"
auditEventScp = "scp"
auditEventResize = "resize"
auditEventShell = "shell"
sshContextSessionID = "sessionID"
sshContextEventLogger = "eventLogger"
sshContextPreamble = "sshPreamble"
sshContextSSHClient = "sshClient"
SSHPreambleLength = 2
defaultSSHPort = "22"
)
type auditEvent struct {
Event string `json:"event,omitempty"`
EventType string `json:"event_type,omitempty"`
SessionID string `json:"session_id,omitempty"`
User string `json:"user,omitempty"`
Login string `json:"login,omitempty"`
Datetime string `json:"datetime,omitempty"`
Hostname string `json:"hostname,omitempty"`
Destination string `json:"destination,omitempty"`
}
// sshConn wraps the incoming net.Conn and a cleanup function
// This is done to allow the outgoing SSH client to be retrieved and closed when the conn itself is closed.
type sshConn struct {
net.Conn
cleanupFunc func()
}
// close calls the cleanupFunc before closing the conn
func (c sshConn) Close() error {
c.cleanupFunc()
return c.Conn.Close()
}
type SSHProxy struct {
ssh.Server
logger *logrus.Logger
shutdownC chan struct{}
caCert ssh.PublicKey
getUserFunc func(string) (*User, error)
logManager sshlog.Manager
hostname string
logger *logrus.Logger
shutdownC chan struct{}
caCert ssh.PublicKey
logManager sshlog.Manager
}
func New(logManager sshlog.Manager, logger *logrus.Logger, address string, shutdownC chan struct{}, idleTimeout, maxTimeout time.Duration) (*SSHServer, error) {
currentUser, err := user.Current()
if err != nil {
return nil, err
}
if currentUser.Uid != "0" {
return nil, errors.New("cloudflared SSH server needs to run as root")
type SSHPreamble struct {
Destination string
JWT string
}
// New creates a new SSHProxy and configures its host keys and authentication by the data provided
func New(logManager sshlog.Manager, logger *logrus.Logger, version, localAddress, hostname, hostKeyDir string, shutdownC chan struct{}, idleTimeout, maxTimeout time.Duration) (*SSHProxy, error) {
sshProxy := SSHProxy{
hostname: hostname,
logger: logger,
shutdownC: shutdownC,
logManager: logManager,
}
sshServer := SSHServer{
Server: ssh.Server{Addr: address, MaxTimeout: maxTimeout, IdleTimeout: idleTimeout},
logger: logger,
shutdownC: shutdownC,
getUserFunc: lookupUser,
logManager: logManager,
sshProxy.Server = ssh.Server{
Addr: localAddress,
MaxTimeout: maxTimeout,
IdleTimeout: idleTimeout,
Version: fmt.Sprintf("SSH-2.0-Cloudflare-Access_%s_%s", version, runtime.GOOS),
PublicKeyHandler: sshProxy.proxyAuthCallback,
ConnCallback: sshProxy.connCallback,
ChannelHandlers: map[string]ssh.ChannelHandler{
"default": sshProxy.channelHandler,
},
}
if err := sshServer.configureHostKeys(); err != nil {
if err := sshProxy.configureHostKeys(hostKeyDir); err != nil {
return nil, err
}
sshServer.configureAuthentication()
return &sshServer, nil
return &sshProxy, nil
}
func (s *SSHServer) Start() error {
// Start the SSH proxy listener to start handling SSH connections from clients
func (s *SSHProxy) Start() error {
s.logger.Infof("Starting SSH server at %s", s.Addr)
go func() {
@@ -66,162 +116,375 @@ func (s *SSHServer) Start() error {
}
}()
s.Handle(s.connectionHandler)
return s.ListenAndServe()
}
func (s *SSHServer) connectionHandler(session ssh.Session) {
sessionID, err := uuid.NewRandom()
// proxyAuthCallback attempts to connect to ultimate SSH destination. If successful, it allows the incoming connection
// to connect to the proxy and saves the outgoing SSH client to the context. Otherwise, no connection to the
// the proxy is allowed.
func (s *SSHProxy) proxyAuthCallback(ctx ssh.Context, key ssh.PublicKey) bool {
client, err := s.dialDestination(ctx)
if err != nil {
if _, err := io.WriteString(session, "Failed to generate session ID\n"); err != nil {
s.logger.WithError(err).Error("Failed to generate session ID: Failed to write to SSH session")
return false
}
ctx.SetValue(sshContextSSHClient, client)
return true
}
// connCallback reads the preamble sent from the proxy server and saves an audit event logger to the context.
// If any errors occur, the connection is terminated by returning nil from the callback.
func (s *SSHProxy) connCallback(ctx ssh.Context, conn net.Conn) net.Conn {
// AUTH-2050: This is a temporary workaround of a timing issue in the tunnel muxer to allow further testing.
// TODO: Remove this
time.Sleep(10 * time.Millisecond)
preamble, err := s.readPreamble(conn)
if err != nil {
if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
s.logger.Warn("Could not establish session. Client likely does not have --destination set and is using old-style ssh config")
} else if err != io.EOF {
s.logger.WithError(err).Error("failed to read SSH preamble")
}
s.CloseSession(session)
return nil
}
ctx.SetValue(sshContextPreamble, preamble)
// Get uid and gid of user attempting to login
sshUser, ok := session.Context().Value("sshUser").(*User)
if !ok || sshUser == nil {
s.logger.Error("Error retrieving credentials from session")
s.CloseSession(session)
return
}
uidInt, err := stringToUint32(sshUser.Uid)
logger, sessionID, err := s.auditLogger()
if err != nil {
s.logger.WithError(err).Error("Invalid user")
s.CloseSession(session)
return
}
gidInt, err := stringToUint32(sshUser.Gid)
if err != nil {
s.logger.WithError(err).Error("Invalid user group")
s.CloseSession(session)
return
s.logger.WithError(err).Error("failed to configure logger")
return nil
}
ctx.SetValue(sshContextEventLogger, logger)
ctx.SetValue(sshContextSessionID, sessionID)
// Spawn shell under user
var cmd *exec.Cmd
if session.RawCommand() != "" {
cmd = exec.Command(sshUser.Shell, "-c", session.RawCommand())
} else {
cmd = exec.Command(sshUser.Shell)
}
// Supplementary groups are not explicitly specified. They seem to be inherited by default.
cmd.SysProcAttr = &syscall.SysProcAttr{Credential: &syscall.Credential{Uid: uidInt, Gid: gidInt}, Setsid: true}
cmd.Env = append(cmd.Env, session.Environ()...)
cmd.Env = append(cmd.Env, fmt.Sprintf("USER=%s", sshUser.Username))
cmd.Env = append(cmd.Env, fmt.Sprintf("HOME=%s", sshUser.HomeDir))
cmd.Dir = sshUser.HomeDir
ptyReq, winCh, isPty := session.Pty()
var shellInput io.WriteCloser
var shellOutput io.ReadCloser
if isPty {
cmd.Env = append(cmd.Env, fmt.Sprintf("TERM=%s", ptyReq.Term))
shellInput, shellOutput, err = s.startPtySession(cmd, winCh)
if err != nil {
s.logger.WithError(err).Error("Failed to start pty session")
close(s.shutdownC)
return
}
} else {
shellInput, shellOutput, err = s.startNonPtySession(cmd)
if err != nil {
s.logger.WithError(err).Error("Failed to start non-pty session")
close(s.shutdownC)
return
// attempts to retrieve and close the outgoing ssh client when the incoming conn is closed.
// If no client exists, the conn is being closed before the PublicKeyCallback was called (where the client is created).
cleanupFunc := func() {
client, ok := ctx.Value(sshContextSSHClient).(*gossh.Client)
if ok && client != nil {
client.Close()
}
}
// Write incoming commands to shell
return sshConn{conn, cleanupFunc}
}
// channelHandler proxies incoming and outgoing SSH traffic back and forth over an SSH Channel
func (s *SSHProxy) channelHandler(srv *ssh.Server, conn *gossh.ServerConn, newChan gossh.NewChannel, ctx ssh.Context) {
if newChan.ChannelType() != "session" && newChan.ChannelType() != "direct-tcpip" {
msg := fmt.Sprintf("channel type %s is not supported", newChan.ChannelType())
s.logger.Info(msg)
if err := newChan.Reject(gossh.UnknownChannelType, msg); err != nil {
s.logger.WithError(err).Error("Error rejecting SSH channel")
}
return
}
localChan, localChanReqs, err := newChan.Accept()
if err != nil {
s.logger.WithError(err).Error("Failed to accept session channel")
return
}
defer localChan.Close()
// client will be closed when the sshConn is closed
client, ok := ctx.Value(sshContextSSHClient).(*gossh.Client)
if !ok {
s.logger.Error("Could not retrieve client from context")
return
}
remoteChan, remoteChanReqs, err := client.OpenChannel(newChan.ChannelType(), newChan.ExtraData())
if err != nil {
s.logger.WithError(err).Error("Failed to open remote channel")
return
}
defer remoteChan.Close()
// Proxy ssh traffic back and forth between client and destination
s.proxyChannel(localChan, remoteChan, localChanReqs, remoteChanReqs, conn, ctx)
}
// proxyChannel couples two SSH channels and proxies SSH traffic and channel requests back and forth.
func (s *SSHProxy) proxyChannel(localChan, remoteChan gossh.Channel, localChanReqs, remoteChanReqs <-chan *gossh.Request, conn *gossh.ServerConn, ctx ssh.Context) {
done := make(chan struct{}, 2)
go func() {
if _, err := io.Copy(shellInput, session); err != nil {
s.logger.WithError(err).Error("Failed to write incoming command to pty")
if _, err := io.Copy(localChan, remoteChan); err != nil {
s.logger.WithError(err).Error("remote to local copy error")
}
done <- struct{}{}
}()
go func() {
if _, err := io.Copy(remoteChan, localChan); err != nil {
s.logger.WithError(err).Error("local to remote copy error")
}
done <- struct{}{}
}()
// stderr streams are used non-pty sessions since they have distinct IO streams.
remoteStderr := remoteChan.Stderr()
localStderr := localChan.Stderr()
go func() {
if _, err := io.Copy(remoteStderr, localStderr); err != nil {
s.logger.WithError(err).Error("stderr local to remote copy error")
}
}()
go func() {
if _, err := io.Copy(localStderr, remoteStderr); err != nil {
s.logger.WithError(err).Error("stderr remote to local copy error")
}
}()
pr, pw := io.Pipe()
defer pr.Close()
defer pw.Close()
s.logAuditEvent(conn, "", auditEventStart, ctx)
defer s.logAuditEvent(conn, "", auditEventStop, ctx)
logger, err := s.logManager.NewLogger(fmt.Sprintf("%s-session.log", sessionID), s.logger)
if err != nil {
if _, err := io.WriteString(session, "Failed to create log\n"); err != nil {
s.logger.WithError(err).Error("Failed to create log: Failed to write to SSH session")
}
s.CloseSession(session)
}
defer logger.Close()
go func() {
io.Copy(logger, pr)
}()
// Write outgoing command output to both the command recorder, and remote user
mw := io.MultiWriter(pw, session)
if _, err := io.Copy(mw, shellOutput); err != nil {
s.logger.WithError(err).Error("Failed to write command output to user")
}
// Wait for all resources associated with cmd to be released
// Returns error if shell exited with a non-zero status or received a signal
if err := cmd.Wait(); err != nil {
s.logger.WithError(err).Debug("Shell did not close correctly")
}
}
func (s *SSHServer) CloseSession(session ssh.Session) {
if err := session.Exit(1); err != nil {
s.logger.WithError(err).Error("Failed to close SSH session")
}
}
func (s *SSHServer) startNonPtySession(cmd *exec.Cmd) (io.WriteCloser, io.ReadCloser, error) {
in, err := cmd.StdinPipe()
if err != nil {
return nil, nil, err
}
out, err := cmd.StdoutPipe()
if err != nil {
return nil, nil, err
}
cmd.Stderr = cmd.Stdout
if err = cmd.Start(); err != nil {
return nil, nil, err
}
return in, out, nil
}
func (s *SSHServer) startPtySession(cmd *exec.Cmd, winCh <-chan ssh.Window) (io.WriteCloser, io.ReadCloser, error) {
tty, err := pty.Start(cmd)
if err != nil {
return nil, nil, err
}
// Handle terminal window size changes
go func() {
for win := range winCh {
if errNo := setWinsize(tty, win.Width, win.Height); errNo != 0 {
s.logger.WithError(err).Error("Failed to set pty window size")
close(s.shutdownC)
// Proxy channel requests
for {
select {
case req := <-localChanReqs:
if req == nil {
return
}
if err := s.forwardChannelRequest(remoteChan, req); err != nil {
s.logger.WithError(err).Error("Failed to forward request")
return
}
s.logChannelRequest(req, conn, ctx)
case req := <-remoteChanReqs:
if req == nil {
return
}
if err := s.forwardChannelRequest(localChan, req); err != nil {
s.logger.WithError(err).Error("Failed to forward request")
return
}
case <-done:
return
}
}
}
// readPreamble reads a preamble from the SSH connection before any SSH traffic is sent.
// This preamble is a JSON encoded struct containing the users JWT and ultimate destination.
// The first 4 bytes contain the length of the preamble which follows immediately.
func (s *SSHProxy) readPreamble(conn net.Conn) (*SSHPreamble, error) {
// Set conn read deadline while reading preamble to prevent hangs if preamble wasnt sent.
if err := conn.SetReadDeadline(time.Now().Add(500 * time.Millisecond)); err != nil {
return nil, errors.Wrap(err, "failed to set conn deadline")
}
defer func() {
if err := conn.SetReadDeadline(time.Time{}); err != nil {
s.logger.WithError(err).Error("Failed to unset conn read deadline")
}
}()
return tty, tty, nil
size := make([]byte, SSHPreambleLength)
if _, err := io.ReadFull(conn, size); err != nil {
return nil, err
}
payloadLength := binary.BigEndian.Uint16(size)
payload := make([]byte, payloadLength)
if _, err := io.ReadFull(conn, payload); err != nil {
return nil, err
}
var preamble SSHPreamble
err := json.Unmarshal(payload, &preamble)
if err != nil {
return nil, err
}
preamble.Destination, err = canonicalizeDest(preamble.Destination)
if err != nil {
return nil, err
}
return &preamble, nil
}
// Sets PTY window size for terminal
func setWinsize(f *os.File, w, h int) syscall.Errno {
_, _, errNo := syscall.Syscall(syscall.SYS_IOCTL, f.Fd(), uintptr(syscall.TIOCSWINSZ),
uintptr(unsafe.Pointer(&struct{ h, w, x, y uint16 }{uint16(h), uint16(w), 0, 0})))
return errNo
// canonicalizeDest adds a default port if one doesnt exist
func canonicalizeDest(dest string) (string, error) {
_, _, err := net.SplitHostPort(dest)
// if host and port are split without error, a port exists.
if err != nil {
addrErr, ok := err.(*net.AddrError)
if !ok {
return "", err
}
// If the port is missing, append it.
if addrErr.Err == "missing port in address" {
return fmt.Sprintf("%s:%s", dest, defaultSSHPort), nil
}
// If there are too many colons and address is IPv6, wrap in brackets and append port. Otherwise invalid address
ip := net.ParseIP(dest)
if addrErr.Err == "too many colons in address" && ip != nil && ip.To4() == nil {
return fmt.Sprintf("[%s]:%s", dest, defaultSSHPort), nil
}
return "", addrErr
}
return dest, nil
}
func stringToUint32(str string) (uint32, error) {
uid, err := strconv.ParseUint(str, 10, 32)
return uint32(uid), err
// dialDestination creates a new SSH client and dials the destination server
func (s *SSHProxy) dialDestination(ctx ssh.Context) (*gossh.Client, error) {
preamble, ok := ctx.Value(sshContextPreamble).(*SSHPreamble)
if !ok {
msg := "failed to retrieve SSH preamble from context"
s.logger.Error(msg)
return nil, errors.New(msg)
}
signer, err := s.genSSHSigner(preamble.JWT)
if err != nil {
s.logger.WithError(err).Error("Failed to generate signed short lived cert")
return nil, err
}
s.logger.Debugf("Short lived certificate for %s connecting to %s:\n\n%s", ctx.User(), preamble.Destination, gossh.MarshalAuthorizedKey(signer.PublicKey()))
clientConfig := &gossh.ClientConfig{
User: ctx.User(),
// AUTH-2103 TODO: proper host key check
HostKeyCallback: gossh.InsecureIgnoreHostKey(),
Auth: []gossh.AuthMethod{gossh.PublicKeys(signer)},
ClientVersion: ctx.ServerVersion(),
}
client, err := gossh.Dial("tcp", preamble.Destination, clientConfig)
if err != nil {
s.logger.WithError(err).Info("Failed to connect to destination SSH server")
return nil, err
}
return client, nil
}
// Generates a key pair and sends public key to get signed by CA
func (s *SSHProxy) genSSHSigner(jwt string) (gossh.Signer, error) {
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
return nil, errors.Wrap(err, "failed to generate ecdsa key pair")
}
pub, err := gossh.NewPublicKey(&key.PublicKey)
if err != nil {
return nil, errors.Wrap(err, "failed to convert ecdsa public key to SSH public key")
}
pubBytes := gossh.MarshalAuthorizedKey(pub)
signedCertBytes, err := sshgen.SignCert(jwt, string(pubBytes))
if err != nil {
return nil, errors.Wrap(err, "failed to retrieve cert from SSHCAAPI")
}
signedPub, _, _, _, err := gossh.ParseAuthorizedKey([]byte(signedCertBytes))
if err != nil {
return nil, errors.Wrap(err, "failed to parse SSH public key")
}
cert, ok := signedPub.(*gossh.Certificate)
if !ok {
return nil, errors.Wrap(err, "failed to assert public key as certificate")
}
signer, err := gossh.NewSignerFromKey(key)
if err != nil {
return nil, errors.Wrap(err, "failed to create signer")
}
certSigner, err := gossh.NewCertSigner(cert, signer)
if err != nil {
return nil, errors.Wrap(err, "failed to create cert signer")
}
return certSigner, nil
}
// forwardChannelRequest sends request req to SSH channel sshChan, waits for reply, and sends the reply back.
func (s *SSHProxy) forwardChannelRequest(sshChan gossh.Channel, req *gossh.Request) error {
reply, err := sshChan.SendRequest(req.Type, req.WantReply, req.Payload)
if err != nil {
return errors.Wrap(err, "Failed to send request")
}
if err := req.Reply(reply, nil); err != nil {
return errors.Wrap(err, "Failed to reply to request")
}
return nil
}
// logChannelRequest creates an audit log for different types of channel requests
func (s *SSHProxy) logChannelRequest(req *gossh.Request, conn *gossh.ServerConn, ctx ssh.Context) {
var eventType string
var event string
switch req.Type {
case "exec":
var payload struct{ Value string }
if err := gossh.Unmarshal(req.Payload, &payload); err != nil {
s.logger.WithError(err).Errorf("Failed to unmarshal channel request payload: %s:%s", req.Type, req.Payload)
}
event = payload.Value
eventType = auditEventExec
if strings.HasPrefix(string(req.Payload), "scp") {
eventType = auditEventScp
}
case "shell":
eventType = auditEventShell
case "window-change":
eventType = auditEventResize
default:
return
}
s.logAuditEvent(conn, event, eventType, ctx)
}
func (s *SSHProxy) auditLogger() (io.WriteCloser, string, error) {
sessionUUID, err := uuid.NewRandom()
if err != nil {
return nil, "", errors.Wrap(err, "failed to create sessionID")
}
sessionID := sessionUUID.String()
writer, err := s.logManager.NewLogger(fmt.Sprintf("%s-event.log", sessionID), s.logger)
if err != nil {
return nil, "", errors.Wrap(err, "failed to create logger")
}
return writer, sessionID, nil
}
func (s *SSHProxy) logAuditEvent(conn *gossh.ServerConn, event, eventType string, ctx ssh.Context) {
sessionID, sessionIDOk := ctx.Value(sshContextSessionID).(string)
writer, writerOk := ctx.Value(sshContextEventLogger).(io.WriteCloser)
if !writerOk || !sessionIDOk {
s.logger.Error("Failed to retrieve audit logger from context")
return
}
var destination string
preamble, ok := ctx.Value(sshContextPreamble).(*SSHPreamble)
if ok {
destination = preamble.Destination
} else {
s.logger.Error("Failed to retrieve SSH preamble from context")
}
ae := auditEvent{
Event: event,
EventType: eventType,
SessionID: sessionID,
User: conn.User(),
Login: conn.User(),
Datetime: time.Now().UTC().Format(time.RFC3339),
Hostname: s.hostname,
Destination: destination,
}
data, err := json.Marshal(&ae)
if err != nil {
s.logger.WithError(err).Error("Failed to marshal audit event. malformed audit object")
return
}
line := string(data) + "\n"
if _, err := writer.Write([]byte(line)); err != nil {
s.logger.WithError(err).Error("Failed to write audit event.")
}
}
+8 -1
View File
@@ -11,9 +11,16 @@ import (
"github.com/sirupsen/logrus"
)
const SSHPreambleLength = 2
type SSHServer struct{}
func New(_ sshlog.Manager, _ *logrus.Logger, _ string, _ chan struct{}, _, _ time.Duration) (*SSHServer, error) {
type SSHPreamble struct {
Destination string
JWT string
}
func New(_ sshlog.Manager, _ *logrus.Logger, _, _, _, _ string, _ chan struct{}, _, _ time.Duration) (*SSHServer, error) {
return nil, errors.New("cloudflared ssh server is not supported on windows")
}
+31 -8
View File
@@ -35,26 +35,49 @@ func createRequest(stream *h2mux.MuxedStream, url *url.URL) (*http.Request, erro
return req, nil
}
// H2RequestHeadersToH1Request converts the HTTP/2 headers to an HTTP/1 Request
// object. This includes conversion of the pseudo-headers into their closest
// HTTP/1 equivalents. See https://tools.ietf.org/html/rfc7540#section-8.1.2.3
func H2RequestHeadersToH1Request(h2 []h2mux.Header, h1 *http.Request) error {
for _, header := range h2 {
switch header.Name {
case ":method":
h1.Method = header.Value
case ":scheme":
// noop - use the preexisting scheme from h1.URL
case ":authority":
// Otherwise the host header will be based on the origin URL
h1.Host = header.Value
case ":path":
u, err := url.Parse(header.Value)
// We don't want to be an "opinionated" proxy, so ideally we would use :path as-is.
// However, this HTTP/1 Request object belongs to the Go standard library,
// whose URL package makes some opinionated decisions about the encoding of
// URL characters: see the docs of https://godoc.org/net/url#URL,
// in particular the EscapedPath method https://godoc.org/net/url#URL.EscapedPath,
// which is always used when computing url.URL.String(), whether we'd like it or not.
//
// Well, not *always*. We could circumvent this by using url.URL.Opaque. But
// that would present unusual difficulties when using an HTTP proxy: url.URL.Opaque
// is treated differently when HTTP_PROXY is set!
// See https://github.com/golang/go/issues/5684#issuecomment-66080888
//
// This means we are subject to the behavior of net/url's function `shouldEscape`
// (as invoked with mode=encodePath): https://github.com/golang/go/blob/go1.12.7/src/net/url/url.go#L101
if header.Value == "*" {
h1.URL.Path = "*"
continue
}
// Due to the behavior of validation.ValidateUrl, h1.URL may
// already have a partial value, with or without a trailing slash.
base := h1.URL.String()
base = strings.TrimRight(base, "/")
// But we know :path begins with '/', because we handled '*' above - see RFC7540
url, err := url.Parse(base + header.Value)
if err != nil {
return fmt.Errorf("unparseable path")
return errors.Wrap(err, fmt.Sprintf("invalid path '%v'", header.Value))
}
resolved := h1.URL.ResolveReference(u)
// prevent escaping base URL
if !strings.HasPrefix(resolved.String(), h1.URL.String()) {
return fmt.Errorf("invalid path")
}
h1.URL = resolved
h1.URL = url
case "content-length":
contentLength, err := strconv.ParseInt(header.Value, 10, 64)
if err != nil {
+441
View File
@@ -0,0 +1,441 @@
package streamhandler
import (
"fmt"
"math/rand"
"net/http"
"net/url"
"reflect"
"regexp"
"strings"
"testing"
"testing/quick"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestH2RequestHeadersToH1Request_RegularHeaders(t *testing.T) {
request, err := http.NewRequest(http.MethodGet, "http://example.com", nil)
assert.NoError(t, err)
headersConversionErr := H2RequestHeadersToH1Request(
[]h2mux.Header{
h2mux.Header{
Name: "Mock header 1",
Value: "Mock value 1",
},
h2mux.Header{
Name: "Mock header 2",
Value: "Mock value 2",
},
},
request,
)
assert.Equal(t, http.Header{
"Mock header 1": []string{"Mock value 1"},
"Mock header 2": []string{"Mock value 2"},
}, request.Header)
assert.NoError(t, headersConversionErr)
}
func TestH2RequestHeadersToH1Request_NoHeaders(t *testing.T) {
request, err := http.NewRequest(http.MethodGet, "http://example.com", nil)
assert.NoError(t, err)
headersConversionErr := H2RequestHeadersToH1Request(
[]h2mux.Header{},
request,
)
assert.Equal(t, http.Header{}, request.Header)
assert.NoError(t, headersConversionErr)
}
func TestH2RequestHeadersToH1Request_InvalidHostPath(t *testing.T) {
request, err := http.NewRequest(http.MethodGet, "http://example.com", nil)
assert.NoError(t, err)
headersConversionErr := H2RequestHeadersToH1Request(
[]h2mux.Header{
h2mux.Header{
Name: ":path",
Value: "//bad_path/",
},
h2mux.Header{
Name: "Mock header",
Value: "Mock value",
},
},
request,
)
assert.Equal(t, http.Header{
"Mock header": []string{"Mock value"},
}, request.Header)
assert.Equal(t, "http://example.com//bad_path/", request.URL.String())
assert.NoError(t, headersConversionErr)
}
func TestH2RequestHeadersToH1Request_HostPathWithQuery(t *testing.T) {
request, err := http.NewRequest(http.MethodGet, "http://example.com/", nil)
assert.NoError(t, err)
headersConversionErr := H2RequestHeadersToH1Request(
[]h2mux.Header{
h2mux.Header{
Name: ":path",
Value: "/?query=mock%20value",
},
h2mux.Header{
Name: "Mock header",
Value: "Mock value",
},
},
request,
)
assert.Equal(t, http.Header{
"Mock header": []string{"Mock value"},
}, request.Header)
assert.Equal(t, "http://example.com/?query=mock%20value", request.URL.String())
assert.NoError(t, headersConversionErr)
}
func TestH2RequestHeadersToH1Request_HostPathWithURLEncoding(t *testing.T) {
request, err := http.NewRequest(http.MethodGet, "http://example.com/", nil)
assert.NoError(t, err)
headersConversionErr := H2RequestHeadersToH1Request(
[]h2mux.Header{
h2mux.Header{
Name: ":path",
Value: "/mock%20path",
},
h2mux.Header{
Name: "Mock header",
Value: "Mock value",
},
},
request,
)
assert.Equal(t, http.Header{
"Mock header": []string{"Mock value"},
}, request.Header)
assert.Equal(t, "http://example.com/mock%20path", request.URL.String())
assert.NoError(t, headersConversionErr)
}
func TestH2RequestHeadersToH1Request_WeirdURLs(t *testing.T) {
type testCase struct {
path string
want string
}
testCases := []testCase{
{
path: "",
want: "",
},
{
path: "/",
want: "/",
},
{
path: "//",
want: "//",
},
{
path: "/test",
want: "/test",
},
{
path: "//test",
want: "//test",
},
{
// https://github.com/cloudflare/cloudflared/issues/81
path: "//test/",
want: "//test/",
},
{
path: "/%2Ftest",
want: "/%2Ftest",
},
{
path: "//%20test",
want: "//%20test",
},
{
// https://github.com/cloudflare/cloudflared/issues/124
path: "/test?get=somthing%20a",
want: "/test?get=somthing%20a",
},
{
path: "/%20",
want: "/%20",
},
{
// stdlib's EscapedPath() will always percent-encode ' '
path: "/ ",
want: "/%20",
},
{
path: "/ a ",
want: "/%20a%20",
},
{
path: "/a%20b",
want: "/a%20b",
},
{
path: "/foo/bar;param?query#frag",
want: "/foo/bar;param?query#frag",
},
{
// stdlib's EscapedPath() will always percent-encode non-ASCII chars
path: "/a␠b",
want: "/a%E2%90%A0b",
},
{
path: "/a-umlaut-ä",
want: "/a-umlaut-%C3%A4",
},
{
path: "/a-umlaut-%C3%A4",
want: "/a-umlaut-%C3%A4",
},
{
path: "/a-umlaut-%c3%a4",
want: "/a-umlaut-%c3%a4",
},
{
// here the second '#' is treated as part of the fragment
path: "/a#b#c",
want: "/a#b%23c",
},
{
path: "/a#b␠c",
want: "/a#b%E2%90%A0c",
},
{
path: "/a#b%20c",
want: "/a#b%20c",
},
{
path: "/a#b c",
want: "/a#b%20c",
},
{
// stdlib's EscapedPath() will always percent-encode '\'
path: "/\\",
want: "/%5C",
},
{
path: "/a\\",
want: "/a%5C",
},
{
path: "/a,b.c.",
want: "/a,b.c.",
},
{
path: "/.",
want: "/.",
},
{
// stdlib's EscapedPath() will always percent-encode '`'
path: "/a`",
want: "/a%60",
},
{
path: "/a[0]",
want: "/a[0]",
},
{
path: "/?a[0]=5 &b[]=",
want: "/?a[0]=5 &b[]=",
},
{
path: "/?a=%22b%20%22",
want: "/?a=%22b%20%22",
},
}
for index, testCase := range testCases {
requestURL := "https://example.com"
request, err := http.NewRequest(http.MethodGet, requestURL, nil)
assert.NoError(t, err)
headersConversionErr := H2RequestHeadersToH1Request(
[]h2mux.Header{
h2mux.Header{
Name: ":path",
Value: testCase.path,
},
h2mux.Header{
Name: "Mock header",
Value: "Mock value",
},
},
request,
)
assert.NoError(t, headersConversionErr)
assert.Equal(t,
http.Header{
"Mock header": []string{"Mock value"},
},
request.Header)
assert.Equal(t,
"https://example.com"+testCase.want,
request.URL.String(),
"Failed URL index: %v %#v", index, testCase)
}
}
func TestH2RequestHeadersToH1Request_QuickCheck(t *testing.T) {
config := &quick.Config{
Values: func(args []reflect.Value, rand *rand.Rand) {
args[0] = reflect.ValueOf(randomHTTP2Path(t, rand))
},
}
type testOrigin struct {
url string
expectedScheme string
expectedBasePath string
}
testOrigins := []testOrigin{
{
url: "http://origin.hostname.example.com:8080",
expectedScheme: "http",
expectedBasePath: "http://origin.hostname.example.com:8080",
},
{
url: "http://origin.hostname.example.com:8080/",
expectedScheme: "http",
expectedBasePath: "http://origin.hostname.example.com:8080",
},
{
url: "http://origin.hostname.example.com:8080/api",
expectedScheme: "http",
expectedBasePath: "http://origin.hostname.example.com:8080/api",
},
{
url: "http://origin.hostname.example.com:8080/api/",
expectedScheme: "http",
expectedBasePath: "http://origin.hostname.example.com:8080/api",
},
{
url: "https://origin.hostname.example.com:8080/api",
expectedScheme: "https",
expectedBasePath: "https://origin.hostname.example.com:8080/api",
},
}
// use multiple schemes to demonstrate that the URL is based on the
// origin's scheme, not the :scheme header
for _, testScheme := range []string{"http", "https"} {
for _, testOrigin := range testOrigins {
assertion := func(testPath string) bool {
const expectedMethod = "POST"
const expectedHostname = "request.hostname.example.com"
h2 := []h2mux.Header{
h2mux.Header{Name: ":method", Value: expectedMethod},
h2mux.Header{Name: ":scheme", Value: testScheme},
h2mux.Header{Name: ":authority", Value: expectedHostname},
h2mux.Header{Name: ":path", Value: testPath},
}
h1, err := http.NewRequest("GET", testOrigin.url, nil)
require.NoError(t, err)
err = H2RequestHeadersToH1Request(h2, h1)
return assert.NoError(t, err) &&
assert.Equal(t, expectedMethod, h1.Method) &&
assert.Equal(t, expectedHostname, h1.Host) &&
assert.Equal(t, testOrigin.expectedScheme, h1.URL.Scheme) &&
assert.Equal(t, testOrigin.expectedBasePath+testPath, h1.URL.String())
}
err := quick.Check(assertion, config)
assert.NoError(t, err)
}
}
}
func randomASCIIPrintableChar(rand *rand.Rand) int {
// smallest printable ASCII char is 32, largest is 126
const startPrintable = 32
const endPrintable = 127
return startPrintable + rand.Intn(endPrintable-startPrintable)
}
// randomASCIIText generates an ASCII string, some of whose characters may be
// percent-encoded. Its "logical length" (ignoring percent-encoding) is
// between 1 and `maxLength`.
func randomASCIIText(rand *rand.Rand, minLength int, maxLength int) string {
length := minLength + rand.Intn(maxLength)
result := ""
for i := 0; i < length; i++ {
c := randomASCIIPrintableChar(rand)
// 1/4 chance of using percent encoding when not necessary
if c == '%' || rand.Intn(4) == 0 {
result += fmt.Sprintf("%%%02X", c)
} else {
result += string(c)
}
}
return result
}
// Calls `randomASCIIText` and ensures the result is a valid URL path,
// i.e. one that can pass unchanged through url.URL.String()
func randomHTTP1Path(t *testing.T, rand *rand.Rand, minLength int, maxLength int) string {
text := randomASCIIText(rand, minLength, maxLength)
regexp, err := regexp.Compile("[^/;,]*")
require.NoError(t, err)
return "/" + regexp.ReplaceAllStringFunc(text, url.PathEscape)
}
// Calls `randomASCIIText` and ensures the result is a valid URL query,
// i.e. one that can pass unchanged through url.URL.String()
func randomHTTP1Query(t *testing.T, rand *rand.Rand, minLength int, maxLength int) string {
text := randomASCIIText(rand, minLength, maxLength)
return "?" + strings.ReplaceAll(text, "#", "%23")
}
// Calls `randomASCIIText` and ensures the result is a valid URL fragment,
// i.e. one that can pass unchanged through url.URL.String()
func randomHTTP1Fragment(t *testing.T, rand *rand.Rand, minLength int, maxLength int) string {
text := randomASCIIText(rand, minLength, maxLength)
url, err := url.Parse("#" + text)
require.NoError(t, err)
return url.String()
}
// Assemble a random :path pseudoheader that is legal by Go stdlib standards
// (i.e. all characters will satisfy "net/url".shouldEscape for their respective locations)
func randomHTTP2Path(t *testing.T, rand *rand.Rand) string {
result := randomHTTP1Path(t, rand, 1, 64)
if rand.Intn(2) == 1 {
result += randomHTTP1Query(t, rand, 1, 32)
}
if rand.Intn(2) == 1 {
result += randomHTTP1Fragment(t, rand, 1, 16)
}
return result
}
+1 -1
View File
@@ -93,7 +93,7 @@ func (s *StreamHandler) UpdateConfig(newConfig []*pogs.ReverseProxyConfig) (fail
toAdd := s.tunnelHostnameMapper.ToAdd(newConfig)
for _, tunnelConfig := range toAdd {
tunnelHostname := tunnelConfig.TunnelHostname
originSerice, err := tunnelConfig.OriginConfigJSONHandler.OriginConfig.Service()
originSerice, err := tunnelConfig.OriginConfig.Service()
if err != nil {
s.logger.WithField("tunnelHostname", tunnelHostname).WithError(err).Error("Invalid origin service config")
failedConfigs = append(failedConfigs, &pogs.FailedConfig{
+9 -13
View File
@@ -49,10 +49,8 @@ func TestServeRequest(t *testing.T) {
reverseProxyConfigs := []*pogs.ReverseProxyConfig{
{
TunnelHostname: testTunnelHostname,
OriginConfigJSONHandler: &pogs.OriginConfigJSONHandler{
OriginConfig: &pogs.HTTPOriginConfig{
URLString: httpServer.URL,
},
OriginConfig: &pogs.HTTPOriginConfig{
URLString: httpServer.URL,
},
},
}
@@ -99,10 +97,8 @@ func TestServeBadRequest(t *testing.T) {
reverseProxyConfigs := []*pogs.ReverseProxyConfig{
{
TunnelHostname: testTunnelHostname,
OriginConfigJSONHandler: &pogs.OriginConfigJSONHandler{
OriginConfig: &pogs.HTTPOriginConfig{
URLString: "",
},
OriginConfig: &pogs.HTTPOriginConfig{
URLString: "",
},
},
}
@@ -145,7 +141,7 @@ type DefaultMuxerPair struct {
doneC chan struct{}
}
func NewDefaultMuxerPair(t assert.TestingT, h h2mux.MuxedStreamHandler) *DefaultMuxerPair {
func NewDefaultMuxerPair(t *testing.T, h h2mux.MuxedStreamHandler) *DefaultMuxerPair {
origin, edge := net.Pipe()
p := &DefaultMuxerPair{
OriginMuxConfig: h2mux.MuxerConfig{
@@ -171,20 +167,20 @@ func NewDefaultMuxerPair(t assert.TestingT, h h2mux.MuxedStreamHandler) *Default
EdgeConn: edge,
doneC: make(chan struct{}),
}
assert.NoError(t, p.Handshake())
assert.NoError(t, p.Handshake(t.Name()))
return p
}
func (p *DefaultMuxerPair) Handshake() error {
func (p *DefaultMuxerPair) Handshake(testName string) error {
ctx, cancel := context.WithTimeout(context.Background(), testHandshakeTimeout)
defer cancel()
errGroup, _ := errgroup.WithContext(ctx)
errGroup.Go(func() (err error) {
p.EdgeMux, err = h2mux.Handshake(p.EdgeConn, p.EdgeConn, p.EdgeMuxConfig)
p.EdgeMux, err = h2mux.Handshake(p.EdgeConn, p.EdgeConn, p.EdgeMuxConfig, h2mux.NewActiveStreamsMetrics(testName, "edge"))
return errors.Wrap(err, "edge handshake failure")
})
errGroup.Go(func() (err error) {
p.OriginMux, err = h2mux.Handshake(p.OriginConn, p.OriginConn, p.OriginMuxConfig)
p.OriginMux, err = h2mux.Handshake(p.OriginConn, p.OriginConn, p.OriginMuxConfig, h2mux.NewActiveStreamsMetrics(testName, "origin"))
return errors.Wrap(err, "origin handshake failure")
})
+24
View File
@@ -16,6 +16,7 @@ import (
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/streamhandler"
"github.com/cloudflare/cloudflared/tunnelrpc/pogs"
"github.com/prometheus/client_golang/prometheus"
"github.com/sirupsen/logrus"
)
@@ -28,6 +29,27 @@ type Supervisor struct {
useConfigResultChan chan<- *pogs.UseConfigurationResult
state *state
logger *logrus.Entry
metrics metrics
}
type metrics struct {
configVersion prometheus.Gauge
}
func newMetrics() metrics {
configVersion := prometheus.NewGauge(prometheus.GaugeOpts{
Namespace: "supervisor",
Subsystem: "supervisor",
Name: "config_version",
Help: "Latest configuration version received from Cloudflare",
},
)
prometheus.MustRegister(
configVersion,
)
return metrics{
configVersion: configVersion,
}
}
func NewSupervisor(
@@ -70,6 +92,7 @@ func NewSupervisor(
useConfigResultChan: useConfigResultChan,
state: newState(defaultClientConfig),
logger: logger.WithField("subsystem", "supervisor"),
metrics: newMetrics(),
}, nil
}
@@ -131,6 +154,7 @@ func (s *Supervisor) notifySubsystemsNewConfig(newConfig *pogs.ClientConfig) *po
Success: true,
}
}
s.metrics.configVersion.Set(float64(newConfig.Version))
s.state.updateConfig(newConfig)
var tunnelHostnames []h2mux.TunnelHostname
+3 -6
View File
@@ -5,7 +5,6 @@ import (
"crypto/x509"
"fmt"
"io/ioutil"
"net"
"runtime"
"sync"
@@ -18,6 +17,8 @@ import (
const (
OriginCAPoolFlag = "origin-ca-pool"
CaCertFlag = "cacert"
edgeTLSServerName = "cftunnel.com"
)
// CertReloader can load and reload a TLS certificate from a particular filepath.
@@ -126,7 +127,7 @@ func CreateTunnelConfig(c *cli.Context) (*tls.Config, error) {
rootCAs = append(rootCAs, c.String(CaCertFlag))
}
userConfig := &TLSParameters{RootCAs: rootCAs}
userConfig := &TLSParameters{RootCAs: rootCAs, ServerName: edgeTLSServerName}
tlsConfig, err := GetConfig(userConfig)
if err != nil {
return nil, err
@@ -142,10 +143,6 @@ func CreateTunnelConfig(c *cli.Context) (*tls.Config, error) {
rootCAPool.AddCert(cert)
}
tlsConfig.RootCAs = rootCAPool
tlsConfig.ServerName = "cftunnel.com"
} else if edgeAddrs := c.StringSlice("edge"); len(edgeAddrs) > 0 {
// Set for development environments and for testing specific origintunneld instances
tlsConfig.ServerName, _, _ = net.SplitHostPort(edgeAddrs[0])
}
if tlsConfig.ServerName == "" && !tlsConfig.InsecureSkipVerify {
-2
View File
@@ -1,5 +1,3 @@
// TODO: Remove the above build tag and include this test when we start compiling with Golang 1.10.0+
package tlsconfig
import (
@@ -193,20 +193,20 @@ func TestTunnelHostnameMapper_ToRemove(t *testing.T) {
func sampleConfig1() *pogs.ReverseProxyConfig {
return &pogs.ReverseProxyConfig{
TunnelHostname: "mock.example.com",
OriginConfigJSONHandler: &pogs.OriginConfigJSONHandler{OriginConfig: &pogs.HelloWorldOriginConfig{}},
Retries: 18,
ConnectionTimeout: 5 * time.Second,
CompressionQuality: 3,
TunnelHostname: "mock.example.com",
OriginConfig: &pogs.HelloWorldOriginConfig{},
Retries: 18,
ConnectionTimeout: 5 * time.Second,
CompressionQuality: 3,
}
}
func sampleConfig2() *pogs.ReverseProxyConfig {
return &pogs.ReverseProxyConfig{
TunnelHostname: "mock2.example.com",
OriginConfigJSONHandler: &pogs.OriginConfigJSONHandler{OriginConfig: &pogs.HelloWorldOriginConfig{}},
Retries: 18,
ConnectionTimeout: 5 * time.Second,
CompressionQuality: 3,
TunnelHostname: "mock2.example.com",
OriginConfig: &pogs.HelloWorldOriginConfig{},
Retries: 18,
ConnectionTimeout: 5 * time.Second,
CompressionQuality: 3,
}
}
+108
View File
@@ -0,0 +1,108 @@
package pogs
import (
"fmt"
"time"
)
// AuthenticateResponse is the serialized response from the Authenticate RPC.
// It's a 1:1 representation of the capnp message, so it's not very useful for programmers.
// Instead, you should call the `Outcome()` method to get a programmer-friendly sum type, with one
// case for each possible outcome.
type AuthenticateResponse struct {
PermanentErr string
RetryableErr string
Jwt []byte
HoursUntilRefresh uint8
}
// Outcome turns the deserialized response of Authenticate into a programmer-friendly sum type.
func (ar AuthenticateResponse) Outcome() AuthOutcome {
// If there was a network error, then cloudflared should retry later,
// because origintunneld couldn't prove whether auth was correct or not.
if ar.RetryableErr != "" {
return &AuthUnknown{Err: fmt.Errorf(ar.RetryableErr), HoursUntilRefresh: ar.HoursUntilRefresh}
}
// If the user's authentication was unsuccessful, the server will return an error explaining why.
// cloudflared should fatal with this error.
if ar.PermanentErr != "" {
return &AuthFail{Err: fmt.Errorf(ar.PermanentErr)}
}
// If auth succeeded, return the token and refresh it when instructed.
if ar.PermanentErr == "" && len(ar.Jwt) > 0 {
return &AuthSuccess{Jwt: ar.Jwt, HoursUntilRefresh: ar.HoursUntilRefresh}
}
// Otherwise the state got messed up.
return nil
}
// AuthOutcome is a programmer-friendly sum type denoting the possible outcomes of Authenticate.
//go-sumtype:decl AuthOutcome
type AuthOutcome interface {
isAuthOutcome()
// Serialize into an AuthenticateResponse which can be sent via Capnp
Serialize() AuthenticateResponse
}
// AuthSuccess means the backend successfully authenticated this cloudflared.
type AuthSuccess struct {
Jwt []byte
HoursUntilRefresh uint8
}
// RefreshAfter is how long cloudflared should wait before rerunning Authenticate.
func (ao *AuthSuccess) RefreshAfter() time.Duration {
return hoursToTime(ao.HoursUntilRefresh)
}
// Serialize into an AuthenticateResponse which can be sent via Capnp
func (ao *AuthSuccess) Serialize() AuthenticateResponse {
return AuthenticateResponse{
Jwt: ao.Jwt,
HoursUntilRefresh: ao.HoursUntilRefresh,
}
}
func (ao *AuthSuccess) isAuthOutcome() {}
// AuthFail means this cloudflared has the wrong auth and should exit.
type AuthFail struct {
Err error
}
// Serialize into an AuthenticateResponse which can be sent via Capnp
func (ao *AuthFail) Serialize() AuthenticateResponse {
return AuthenticateResponse{
PermanentErr: ao.Err.Error(),
}
}
func (ao *AuthFail) isAuthOutcome() {}
// AuthUnknown means the backend couldn't finish checking authentication. Try again later.
type AuthUnknown struct {
Err error
HoursUntilRefresh uint8
}
// RefreshAfter is how long cloudflared should wait before rerunning Authenticate.
func (ao *AuthUnknown) RefreshAfter() time.Duration {
return hoursToTime(ao.HoursUntilRefresh)
}
// Serialize into an AuthenticateResponse which can be sent via Capnp
func (ao *AuthUnknown) Serialize() AuthenticateResponse {
return AuthenticateResponse{
RetryableErr: ao.Err.Error(),
HoursUntilRefresh: ao.HoursUntilRefresh,
}
}
func (ao *AuthUnknown) isAuthOutcome() {}
func hoursToTime(hours uint8) time.Duration {
return time.Duration(hours) * time.Hour
}
+78
View File
@@ -0,0 +1,78 @@
package pogs
import (
"context"
"github.com/cloudflare/cloudflared/tunnelrpc"
"zombiezen.com/go/capnproto2/pogs"
"zombiezen.com/go/capnproto2/server"
)
func (i TunnelServer_PogsImpl) Authenticate(p tunnelrpc.TunnelServer_authenticate) error {
originCert, err := p.Params.OriginCert()
if err != nil {
return err
}
hostname, err := p.Params.Hostname()
if err != nil {
return err
}
options, err := p.Params.Options()
if err != nil {
return err
}
pogsOptions, err := UnmarshalRegistrationOptions(options)
if err != nil {
return err
}
server.Ack(p.Options)
resp, err := i.impl.Authenticate(p.Ctx, originCert, hostname, pogsOptions)
if err != nil {
return err
}
result, err := p.Results.NewResult()
if err != nil {
return err
}
return MarshalAuthenticateResponse(result, resp)
}
func MarshalAuthenticateResponse(s tunnelrpc.AuthenticateResponse, p *AuthenticateResponse) error {
return pogs.Insert(tunnelrpc.AuthenticateResponse_TypeID, s.Struct, p)
}
func (c TunnelServer_PogsClient) Authenticate(ctx context.Context, originCert []byte, hostname string, options *RegistrationOptions) (*AuthenticateResponse, error) {
client := tunnelrpc.TunnelServer{Client: c.Client}
promise := client.Authenticate(ctx, func(p tunnelrpc.TunnelServer_authenticate_Params) error {
err := p.SetOriginCert(originCert)
if err != nil {
return err
}
err = p.SetHostname(hostname)
if err != nil {
return err
}
registrationOptions, err := p.NewOptions()
if err != nil {
return err
}
err = MarshalRegistrationOptions(registrationOptions, options)
if err != nil {
return err
}
return nil
})
retval, err := promise.Result().Struct()
if err != nil {
return nil, err
}
return UnmarshalAuthenticateResponse(retval)
}
func UnmarshalAuthenticateResponse(s tunnelrpc.AuthenticateResponse) (*AuthenticateResponse, error) {
p := new(AuthenticateResponse)
err := pogs.Extract(p, tunnelrpc.AuthenticateResponse_TypeID, s.Struct)
return p, err
}
+113
View File
@@ -0,0 +1,113 @@
package pogs
import (
"fmt"
"reflect"
"testing"
"time"
"github.com/cloudflare/cloudflared/tunnelrpc"
"github.com/stretchr/testify/assert"
capnp "zombiezen.com/go/capnproto2"
)
// Ensure the AuthOutcome sum is correct
var _ AuthOutcome = &AuthSuccess{}
var _ AuthOutcome = &AuthFail{}
var _ AuthOutcome = &AuthUnknown{}
// Unit tests for AuthenticateResponse.Outcome()
func TestAuthenticateResponseOutcome(t *testing.T) {
type fields struct {
PermanentErr string
RetryableErr string
Jwt []byte
HoursUntilRefresh uint8
}
tests := []struct {
name string
fields fields
want AuthOutcome
}{
{"success",
fields{Jwt: []byte("asdf"), HoursUntilRefresh: 6},
&AuthSuccess{Jwt: []byte("asdf"), HoursUntilRefresh: 6},
},
{"fail",
fields{PermanentErr: "bad creds"},
&AuthFail{Err: fmt.Errorf("bad creds")},
},
{"error",
fields{RetryableErr: "bad conn", HoursUntilRefresh: 6},
&AuthUnknown{Err: fmt.Errorf("bad conn"), HoursUntilRefresh: 6},
},
{"nil (no fields are set)",
fields{},
nil,
},
{"nil (too few fields are set)",
fields{HoursUntilRefresh: 6},
nil,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ar := AuthenticateResponse{
PermanentErr: tt.fields.PermanentErr,
RetryableErr: tt.fields.RetryableErr,
Jwt: tt.fields.Jwt,
HoursUntilRefresh: tt.fields.HoursUntilRefresh,
}
got := ar.Outcome()
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("AuthenticateResponse.Outcome() = %T, want %v", got, tt.want)
}
if got != nil && !reflect.DeepEqual(got.Serialize(), ar) {
t.Errorf(".Outcome() and .Serialize() should be inverses but weren't. Expected %v, got %v", ar, got.Serialize())
}
})
}
}
func TestWhenToRefresh(t *testing.T) {
expected := 4 * time.Hour
actual := hoursToTime(4)
if expected != actual {
t.Fatalf("expected %v hours, got %v", expected, actual)
}
}
// Test that serializing and deserializing AuthenticationResponse undo each other.
func TestSerializeAuthenticationResponse(t *testing.T) {
tests := []*AuthenticateResponse{
&AuthenticateResponse{
Jwt: []byte("\xbd\xb2\x3d\xbc\x20\xe2\x8c\x98"),
HoursUntilRefresh: 24,
},
&AuthenticateResponse{
PermanentErr: "bad auth",
},
&AuthenticateResponse{
RetryableErr: "bad connection",
HoursUntilRefresh: 24,
},
}
for i, testCase := range tests {
_, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))
capnpEntity, err := tunnelrpc.NewAuthenticateResponse(seg)
if !assert.NoError(t, err) {
t.Fatal("Couldn't initialize a new message")
}
err = MarshalAuthenticateResponse(capnpEntity, testCase)
if !assert.NoError(t, err, "testCase index %v failed to marshal", i) {
continue
}
result, err := UnmarshalAuthenticateResponse(capnpEntity)
if !assert.NoError(t, err, "testCase index %v failed to unmarshal", i) {
continue
}
assert.Equal(t, testCase, result, "testCase index %v didn't preserve struct through marshalling and unmarshalling", i)
}
}
+103 -130
View File
@@ -4,7 +4,6 @@ import (
"context"
"crypto/tls"
"crypto/x509"
"encoding/json"
"fmt"
"net"
"net/http"
@@ -29,11 +28,43 @@ import (
// ClientConfig is a collection of FallibleConfig that determines how cloudflared should function
type ClientConfig struct {
Version Version `json:"version"`
SupervisorConfig *SupervisorConfig `json:"supervisor_config"`
EdgeConnectionConfig *EdgeConnectionConfig `json:"edge_connection_config"`
DoHProxyConfigs []*DoHProxyConfig `json:"doh_proxy_configs" capnp:"dohProxyConfigs"`
ReverseProxyConfigs []*ReverseProxyConfig `json:"reverse_proxy_configs"`
Version Version
SupervisorConfig *SupervisorConfig
EdgeConnectionConfig *EdgeConnectionConfig
DoHProxyConfigs []*DoHProxyConfig `capnp:"dohProxyConfigs"`
ReverseProxyConfigs []*ReverseProxyConfig
}
func (c *ClientConfig) MarshalBytes() ([]byte, error) {
msg, firstSeg, err := capnp.NewMessage(capnp.SingleSegment(nil))
if err != nil {
return nil, err
}
capnpEntity, err := tunnelrpc.NewRootClientConfig(firstSeg)
if err != nil {
return nil, err
}
err = MarshalClientConfig(capnpEntity, c)
if err != nil {
return nil, err
}
return msg.Marshal()
}
func UnmarshalClientConfigFromBytes(clientConfigBytes []byte) (*ClientConfig, error) {
msg, err := capnp.Unmarshal(clientConfigBytes)
if err != nil {
return nil, err
}
capnpClientConfig, err := tunnelrpc.ReadRootClientConfig(msg)
if err != nil {
return nil, err
}
pogsClientConfig, err := UnmarshalClientConfig(capnpClientConfig)
if err != nil {
return nil, err
}
return pogsClientConfig, nil
}
// Version type models the version of a ClientConfig
@@ -52,16 +83,17 @@ func (v Version) String() string {
}
// FallibleConfig is an interface implemented by configs that cloudflared might not be able to apply
//go-sumtype:decl FallibleConfig
type FallibleConfig interface {
FailReason(err error) string
jsonType() string
isFallibleConfig()
}
// SupervisorConfig specifies config of components managed by Supervisor other than ConnectionManager
type SupervisorConfig struct {
AutoUpdateFrequency time.Duration `json:"auto_update_frequency"`
MetricsUpdateFrequency time.Duration `json:"metrics_update_frequency"`
GracePeriod time.Duration `json:"grace_period"`
AutoUpdateFrequency time.Duration
MetricsUpdateFrequency time.Duration
GracePeriod time.Duration
}
// FailReason impelents FallibleConfig interface for SupervisorConfig
@@ -69,23 +101,15 @@ func (sc *SupervisorConfig) FailReason(err error) string {
return fmt.Sprintf("Cannot apply SupervisorConfig, err: %v", err)
}
func (sc *SupervisorConfig) MarshalJSON() ([]byte, error) {
marshaler := make(map[string]SupervisorConfig, 1)
marshaler[sc.jsonType()] = *sc
return json.Marshal(marshaler)
}
func (sc *SupervisorConfig) jsonType() string {
return "supervisor_config"
}
func (_ *SupervisorConfig) isFallibleConfig() {}
// EdgeConnectionConfig specifies what parameters and how may connections should ConnectionManager establish with edge
type EdgeConnectionConfig struct {
NumHAConnections uint8 `json:"num_ha_connections"`
HeartbeatInterval time.Duration `json:"heartbeat_interval"`
Timeout time.Duration `json:"timeout"`
MaxFailedHeartbeats uint64 `json:"max_failed_heartbeats"`
UserCredentialPath string `json:"user_credential_path"`
NumHAConnections uint8
HeartbeatInterval time.Duration
Timeout time.Duration
MaxFailedHeartbeats uint64
UserCredentialPath string
}
// FailReason impelents FallibleConfig interface for EdgeConnectionConfig
@@ -93,21 +117,13 @@ func (cmc *EdgeConnectionConfig) FailReason(err error) string {
return fmt.Sprintf("Cannot apply EdgeConnectionConfig, err: %v", err)
}
func (cmc *EdgeConnectionConfig) MarshalJSON() ([]byte, error) {
marshaler := make(map[string]EdgeConnectionConfig, 1)
marshaler[cmc.jsonType()] = *cmc
return json.Marshal(marshaler)
}
func (cmc *EdgeConnectionConfig) jsonType() string {
return "edge_connection_config"
}
func (_ *EdgeConnectionConfig) isFallibleConfig() {}
// DoHProxyConfig is configuration for DNS over HTTPS service
type DoHProxyConfig struct {
ListenHost string `json:"listen_host"`
ListenPort uint16 `json:"listen_port"`
Upstreams []string `json:"upstreams"`
ListenHost string
ListenPort uint16
Upstreams []string
}
// FailReason impelents FallibleConfig interface for DoHProxyConfig
@@ -115,23 +131,15 @@ func (dpc *DoHProxyConfig) FailReason(err error) string {
return fmt.Sprintf("Cannot apply DoHProxyConfig, err: %v", err)
}
func (dpc *DoHProxyConfig) MarshalJSON() ([]byte, error) {
marshaler := make(map[string]DoHProxyConfig, 1)
marshaler[dpc.jsonType()] = *dpc
return json.Marshal(marshaler)
}
func (dpc *DoHProxyConfig) jsonType() string {
return "doh_proxy_config"
}
func (_ *DoHProxyConfig) isFallibleConfig() {}
// ReverseProxyConfig how and for what hostnames can this cloudflared proxy
type ReverseProxyConfig struct {
TunnelHostname h2mux.TunnelHostname `json:"tunnel_hostname"`
OriginConfigJSONHandler *OriginConfigJSONHandler `json:"origin_config"`
Retries uint64 `json:"retries"`
ConnectionTimeout time.Duration `json:"connection_timeout"`
CompressionQuality uint64 `json:"compression_quality"`
TunnelHostname h2mux.TunnelHostname
OriginConfig OriginConfig
Retries uint64
ConnectionTimeout time.Duration
CompressionQuality uint64
}
func NewReverseProxyConfig(
@@ -145,11 +153,11 @@ func NewReverseProxyConfig(
return nil, fmt.Errorf("NewReverseProxyConfig: originConfigUnmarshaler was null")
}
return &ReverseProxyConfig{
TunnelHostname: h2mux.TunnelHostname(tunnelHostname),
OriginConfigJSONHandler: &OriginConfigJSONHandler{originConfig},
Retries: retries,
ConnectionTimeout: connectionTimeout,
CompressionQuality: compressionQuality,
TunnelHostname: h2mux.TunnelHostname(tunnelHostname),
OriginConfig: originConfig,
Retries: retries,
ConnectionTimeout: connectionTimeout,
CompressionQuality: compressionQuality,
}, nil
}
@@ -158,58 +166,29 @@ func (rpc *ReverseProxyConfig) FailReason(err error) string {
return fmt.Sprintf("Cannot apply ReverseProxyConfig, err: %v", err)
}
func (rpc *ReverseProxyConfig) MarshalJSON() ([]byte, error) {
marshaler := make(map[string]ReverseProxyConfig, 1)
marshaler[rpc.jsonType()] = *rpc
return json.Marshal(marshaler)
}
func (rpc *ReverseProxyConfig) jsonType() string {
return "reverse_proxy_config"
}
func (_ *ReverseProxyConfig) isFallibleConfig() {}
//go-sumtype:decl OriginConfig
type OriginConfig interface {
// Service returns a OriginService used to proxy to the origin
Service() (originservice.OriginService, error)
// go-sumtype requires at least one unexported method, otherwise it will complain that interface is not sealed
jsonType() string
}
type originType int
const (
httpType originType = iota
wsType
helloWorldType
)
func (ot originType) String() string {
switch ot {
case httpType:
return "Http"
case wsType:
return "WebSocket"
case helloWorldType:
return "HelloWorld"
default:
return "unknown"
}
isOriginConfig()
}
type HTTPOriginConfig struct {
URLString string `capnp:"urlString" json:"url_string" mapstructure:"url_string"`
TCPKeepAlive time.Duration `capnp:"tcpKeepAlive" json:"tcp_keep_alive" mapstructure:"tcp_keep_alive"`
DialDualStack bool `json:"dial_dual_stack" mapstructure:"dial_dual_stack"`
TLSHandshakeTimeout time.Duration `capnp:"tlsHandshakeTimeout" json:"tls_handshake_timeout" mapstructure:"tls_handshake_timeout"`
TLSVerify bool `capnp:"tlsVerify" json:"tls_verify" mapstructure:"tls_verify"`
OriginCAPool string `json:"origin_ca_pool" mapstructure:"origin_ca_pool"`
OriginServerName string `json:"origin_server_name" mapstructure:"origin_server_name"`
MaxIdleConnections uint64 `json:"max_idle_connections" mapstructure:"max_idle_connections"`
IdleConnectionTimeout time.Duration `json:"idle_connection_timeout" mapstructure:"idle_connection_timeout"`
ProxyConnectionTimeout time.Duration `json:"proxy_connection_timeout" mapstructure:"proxy_connection_timeout"`
ExpectContinueTimeout time.Duration `json:"expect_continue_timeout" mapstructure:"expect_continue_timeout"`
ChunkedEncoding bool `json:"chunked_encoding" mapstructure:"chunked_encoding"`
URLString string `capnp:"urlString"`
TCPKeepAlive time.Duration `capnp:"tcpKeepAlive"`
DialDualStack bool
TLSHandshakeTimeout time.Duration `capnp:"tlsHandshakeTimeout"`
TLSVerify bool `capnp:"tlsVerify"`
OriginCAPool string
OriginServerName string
MaxIdleConnections uint64
IdleConnectionTimeout time.Duration
ProxyConnectionTimeout time.Duration
ExpectContinueTimeout time.Duration
ChunkedEncoding bool
}
func (hc *HTTPOriginConfig) Service() (originservice.OriginService, error) {
@@ -248,15 +227,13 @@ func (hc *HTTPOriginConfig) Service() (originservice.OriginService, error) {
return originservice.NewHTTPService(transport, url, hc.ChunkedEncoding), nil
}
func (_ *HTTPOriginConfig) jsonType() string {
return httpType.String()
}
func (*HTTPOriginConfig) isOriginConfig() {}
type WebSocketOriginConfig struct {
URLString string `capnp:"urlString" json:"url_string" mapstructure:"url_string"`
TLSVerify bool `capnp:"tlsVerify" json:"tls_verify" mapstructure:"tls_verify"`
OriginCAPool string `json:"origin_ca_pool" mapstructure:"origin_ca_pool"`
OriginServerName string `json:"origin_server_name" mapstructure:"origin_server_name"`
URLString string `capnp:"urlString"`
TLSVerify bool `capnp:"tlsVerify"`
OriginCAPool string
OriginServerName string
}
func (wsc *WebSocketOriginConfig) Service() (originservice.OriginService, error) {
@@ -277,13 +254,11 @@ func (wsc *WebSocketOriginConfig) Service() (originservice.OriginService, error)
return originservice.NewWebSocketService(tlsConfig, url)
}
func (_ *WebSocketOriginConfig) jsonType() string {
return wsType.String()
}
func (*WebSocketOriginConfig) isOriginConfig() {}
type HelloWorldOriginConfig struct{}
func (_ *HelloWorldOriginConfig) Service() (originservice.OriginService, error) {
func (*HelloWorldOriginConfig) Service() (originservice.OriginService, error) {
helloCert, err := tlsconfig.GetHelloCertificateX509()
if err != nil {
return nil, errors.Wrap(err, "Cannot get Hello World server certificate")
@@ -308,9 +283,7 @@ func (_ *HelloWorldOriginConfig) Service() (originservice.OriginService, error)
return originservice.NewHelloWorldService(transport)
}
func (_ *HelloWorldOriginConfig) jsonType() string {
return helloWorldType.String()
}
func (*HelloWorldOriginConfig) isOriginConfig() {}
/*
* Boilerplate to convert between these structs and the primitive structs
@@ -519,9 +492,9 @@ func UnmarshalDoHProxyConfig(s tunnelrpc.DoHProxyConfig) (*DoHProxyConfig, error
func MarshalReverseProxyConfig(s tunnelrpc.ReverseProxyConfig, p *ReverseProxyConfig) error {
s.SetTunnelHostname(p.TunnelHostname.String())
switch config := p.OriginConfigJSONHandler.OriginConfig.(type) {
switch config := p.OriginConfig.(type) {
case *HTTPOriginConfig:
ss, err := s.Origin().NewHttp()
ss, err := s.OriginConfig().NewHttp()
if err != nil {
return err
}
@@ -529,7 +502,7 @@ func MarshalReverseProxyConfig(s tunnelrpc.ReverseProxyConfig, p *ReverseProxyCo
return err
}
case *WebSocketOriginConfig:
ss, err := s.Origin().NewWebsocket()
ss, err := s.OriginConfig().NewWebsocket()
if err != nil {
return err
}
@@ -537,7 +510,7 @@ func MarshalReverseProxyConfig(s tunnelrpc.ReverseProxyConfig, p *ReverseProxyCo
return err
}
case *HelloWorldOriginConfig:
ss, err := s.Origin().NewHelloWorld()
ss, err := s.OriginConfig().NewHelloWorld()
if err != nil {
return err
}
@@ -560,9 +533,9 @@ func UnmarshalReverseProxyConfig(s tunnelrpc.ReverseProxyConfig) (*ReverseProxyC
return nil, err
}
p.TunnelHostname = h2mux.TunnelHostname(tunnelHostname)
switch s.Origin().Which() {
case tunnelrpc.ReverseProxyConfig_origin_Which_http:
ss, err := s.Origin().Http()
switch s.OriginConfig().Which() {
case tunnelrpc.ReverseProxyConfig_originConfig_Which_http:
ss, err := s.OriginConfig().Http()
if err != nil {
return nil, err
}
@@ -570,9 +543,9 @@ func UnmarshalReverseProxyConfig(s tunnelrpc.ReverseProxyConfig) (*ReverseProxyC
if err != nil {
return nil, err
}
p.OriginConfigJSONHandler = &OriginConfigJSONHandler{config}
case tunnelrpc.ReverseProxyConfig_origin_Which_websocket:
ss, err := s.Origin().Websocket()
p.OriginConfig = config
case tunnelrpc.ReverseProxyConfig_originConfig_Which_websocket:
ss, err := s.OriginConfig().Websocket()
if err != nil {
return nil, err
}
@@ -580,9 +553,9 @@ func UnmarshalReverseProxyConfig(s tunnelrpc.ReverseProxyConfig) (*ReverseProxyC
if err != nil {
return nil, err
}
p.OriginConfigJSONHandler = &OriginConfigJSONHandler{config}
case tunnelrpc.ReverseProxyConfig_origin_Which_helloWorld:
ss, err := s.Origin().HelloWorld()
p.OriginConfig = config
case tunnelrpc.ReverseProxyConfig_originConfig_Which_helloWorld:
ss, err := s.OriginConfig().HelloWorld()
if err != nil {
return nil, err
}
@@ -590,7 +563,7 @@ func UnmarshalReverseProxyConfig(s tunnelrpc.ReverseProxyConfig) (*ReverseProxyC
if err != nil {
return nil, err
}
p.OriginConfigJSONHandler = &OriginConfigJSONHandler{config}
p.OriginConfig = config
}
p.Retries = s.Retries()
p.ConnectionTimeout = time.Duration(s.ConnectionTimeout())
@@ -690,13 +663,13 @@ func (i ClientService_PogsImpl) UseConfiguration(p tunnelrpc.ClientService_useCo
}
type UseConfigurationResult struct {
Success bool `json:"success"`
FailedConfigs []*FailedConfig `json:"failed_configs"`
Success bool
FailedConfigs []*FailedConfig
}
type FailedConfig struct {
Config FallibleConfig `json:"config"`
Reason string `json:"reason"`
Config FallibleConfig
Reason string
}
func MarshalFailedConfig(s tunnelrpc.FailedConfig, p *FailedConfig) error {
+19 -26
View File
@@ -61,13 +61,13 @@ func ClientConfigTestCases() []*ClientConfig {
sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
}),
sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{sampleHTTPOriginConfig()}
c.OriginConfig = sampleHTTPOriginConfig()
}),
sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{sampleHTTPOriginConfigUnixPath()}
c.OriginConfig = sampleHTTPOriginConfigUnixPath()
}),
sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{sampleWebSocketOriginConfig()}
c.OriginConfig = sampleWebSocketOriginConfig()
}),
}
}
@@ -83,21 +83,14 @@ func ClientConfigTestCases() []*ClientConfig {
}
func TestClientConfig(t *testing.T) {
for i, testCase := range ClientConfigTestCases() {
_, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))
capnpEntity, err := tunnelrpc.NewClientConfig(seg)
if !assert.NoError(t, err) {
t.Fatal("Couldn't initialize a new message")
}
err = MarshalClientConfig(capnpEntity, testCase)
if !assert.NoError(t, err, "testCase index %v failed to marshal", i) {
continue
}
result, err := UnmarshalClientConfig(capnpEntity)
if !assert.NoError(t, err, "testCase index %v failed to unmarshal", i) {
continue
}
assert.Equal(t, testCase, result, "testCase index %v didn't preserve struct through marshalling and unmarshalling", i)
for _, testCase := range ClientConfigTestCases() {
b, err := testCase.MarshalBytes()
assert.NoError(t, err)
clientConfig, err := UnmarshalClientConfigFromBytes(b)
assert.NoError(t, err)
assert.Equal(t, testCase, clientConfig)
}
}
@@ -167,13 +160,13 @@ func TestReverseProxyConfig(t *testing.T) {
testCases := []*ReverseProxyConfig{
sampleReverseProxyConfig(),
sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{sampleHTTPOriginConfig()}
c.OriginConfig = sampleHTTPOriginConfig()
}),
sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{sampleHTTPOriginConfigUnixPath()}
c.OriginConfig = sampleHTTPOriginConfigUnixPath()
}),
sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{sampleWebSocketOriginConfig()}
c.OriginConfig = sampleWebSocketOriginConfig()
}),
}
for i, testCase := range testCases {
@@ -323,11 +316,11 @@ func sampleDoHProxyConfig(overrides ...func(*DoHProxyConfig)) *DoHProxyConfig {
// applies any number of overrides to it, and returns it.
func sampleReverseProxyConfig(overrides ...func(*ReverseProxyConfig)) *ReverseProxyConfig {
sample := &ReverseProxyConfig{
TunnelHostname: "mock-non-lb-tunnel.example.com",
OriginConfigJSONHandler: &OriginConfigJSONHandler{&HelloWorldOriginConfig{}},
Retries: 18,
ConnectionTimeout: 5 * time.Second,
CompressionQuality: 3,
TunnelHostname: "mock-non-lb-tunnel.example.com",
OriginConfig: &HelloWorldOriginConfig{},
Retries: 18,
ConnectionTimeout: 5 * time.Second,
CompressionQuality: 3,
}
sample.ensureNoZeroFields()
for _, f := range overrides {
-68
View File
@@ -1,68 +0,0 @@
package pogs
import (
"encoding/json"
"fmt"
"github.com/mitchellh/mapstructure"
"github.com/pkg/errors"
)
// OriginConfigJSONHandler is a wrapper to serialize OriginConfig with type information, and deserialize JSON
// into an OriginConfig.
type OriginConfigJSONHandler struct {
OriginConfig OriginConfig
}
func (ocjh *OriginConfigJSONHandler) MarshalJSON() ([]byte, error) {
marshaler := make(map[string]OriginConfig, 1)
marshaler[ocjh.OriginConfig.jsonType()] = ocjh.OriginConfig
return json.Marshal(marshaler)
}
func (ocjh *OriginConfigJSONHandler) UnmarshalJSON(b []byte) error {
var originJSON map[string]interface{}
if err := json.Unmarshal(b, &originJSON); err != nil {
return errors.Wrapf(err, "cannot unmarshal %s into originJSON", string(b))
}
if originConfig, ok := originJSON[httpType.String()]; ok {
httpOriginConfig := &HTTPOriginConfig{}
if err := mapstructure.Decode(originConfig, httpOriginConfig); err != nil {
return errors.Wrapf(err, "cannot decode %+v into HTTPOriginConfig", originConfig)
}
ocjh.OriginConfig = httpOriginConfig
return nil
}
if originConfig, ok := originJSON[wsType.String()]; ok {
wsOriginConfig := &WebSocketOriginConfig{}
if err := mapstructure.Decode(originConfig, wsOriginConfig); err != nil {
return errors.Wrapf(err, "cannot decode %+v into WebSocketOriginConfig", originConfig)
}
ocjh.OriginConfig = wsOriginConfig
return nil
}
if originConfig, ok := originJSON[helloWorldType.String()]; ok {
helloWorldOriginConfig := &HelloWorldOriginConfig{}
if err := mapstructure.Decode(originConfig, helloWorldOriginConfig); err != nil {
return errors.Wrapf(err, "cannot decode %+v into HelloWorldOriginConfig", originConfig)
}
ocjh.OriginConfig = helloWorldOriginConfig
return nil
}
return fmt.Errorf("cannot unmarshal %s into OriginConfig", string(b))
}
// FallibleConfigMarshaler is a wrapper for FallibleConfig to implement custom marshal logic
type FallibleConfigMarshaler struct {
FallibleConfig FallibleConfig
}
func (fcm *FallibleConfigMarshaler) MarshalJSON() ([]byte, error) {
marshaler := make(map[string]FallibleConfig, 1)
marshaler[fcm.FallibleConfig.jsonType()] = fcm.FallibleConfig
return json.Marshal(marshaler)
}
-285
View File
@@ -1,285 +0,0 @@
package pogs
import (
"encoding/json"
"strings"
"testing"
"time"
"github.com/stretchr/testify/assert"
)
func TestUnmarshalOrigin(t *testing.T) {
tests := []struct {
jsonLiteral string
exceptedOriginConfig OriginConfig
}{
{
jsonLiteral: `{
"Http":{
"url_string":"https.example.com",
"tcp_keep_alive":7000000000,
"dial_dual_stack":true,
"tls_handshake_timeout":11000000000,
"tls_verify":true,
"origin_ca_pool":"/etc/cert.pem",
"origin_server_name":"secure.example.com",
"max_idle_connections":19,
"idle_connection_timeout":17000000000,
"proxy_connection_timeout":15000000000,
"expect_continue_timeout":21000000000,
"chunked_encoding":true
}
}`,
exceptedOriginConfig: sampleHTTPOriginConfig(),
},
{
jsonLiteral: `{
"WebSocket":{
"url_string":"ssh://example.com",
"tls_verify":true,
"origin_ca_pool":"/etc/cert.pem",
"origin_server_name":"secure.example.com"
}
}`,
exceptedOriginConfig: sampleWebSocketOriginConfig(),
},
{
jsonLiteral: `{
"HelloWorld": {}
}`,
exceptedOriginConfig: &HelloWorldOriginConfig{},
},
}
for _, test := range tests {
originConfigJSON := prettyToValidJSON(test.jsonLiteral)
var OriginConfigJSONHandler OriginConfigJSONHandler
err := json.Unmarshal([]byte(originConfigJSON), &OriginConfigJSONHandler)
assert.NoError(t, err)
assert.Equal(t, test.exceptedOriginConfig, OriginConfigJSONHandler.OriginConfig)
}
}
func TestUnmarshalClientConfig(t *testing.T) {
prettyClientConfigJSON := `{
"version":10,
"supervisor_config":{
"auto_update_frequency":86400000000000,
"metrics_update_frequency":300000000000,
"grace_period":30000000000
},
"edge_connection_config":{
"num_ha_connections":4,
"heartbeat_interval":5000000000,
"timeout":30000000000,
"max_failed_heartbeats":5,
"user_credential_path":"~/.cloudflared/cert.pem"
},
"doh_proxy_configs":[{
"listen_host": "localhost",
"listen_port": 53,
"upstreams": ["https://1.1.1.1/dns-query", "https://1.0.0.1/dns-query"]
}],
"reverse_proxy_configs":[{
"tunnel_hostname":"sdfjadk33.cftunnel.com",
"origin_config":{
"Http":{
"url_string":"https://127.0.0.1:8080",
"tcp_keep_alive":30000000000,
"dial_dual_stack":true,
"tls_handshake_timeout":10000000000,
"tls_verify":true,
"origin_ca_pool":"",
"origin_server_name":"",
"max_idle_connections":100,
"idle_connection_timeout":90000000000,
"proxy_connection_timeout":90000000000,
"expect_continue_timeout":90000000000,
"chunked_encoding":true
}
},
"retries":5,
"connection_timeout":30,
"compression_quality":0
}]
}`
// replace new line and tab
clientConfigJSON := prettyToValidJSON(prettyClientConfigJSON)
var clientConfig ClientConfig
err := json.Unmarshal([]byte(clientConfigJSON), &clientConfig)
assert.NoError(t, err)
assert.Equal(t, Version(10), clientConfig.Version)
supervisorConfig := SupervisorConfig{
AutoUpdateFrequency: time.Hour * 24,
MetricsUpdateFrequency: time.Second * 300,
GracePeriod: time.Second * 30,
}
assert.Equal(t, supervisorConfig, *clientConfig.SupervisorConfig)
edgeConnectionConfig := EdgeConnectionConfig{
NumHAConnections: 4,
HeartbeatInterval: time.Second * 5,
Timeout: time.Second * 30,
MaxFailedHeartbeats: 5,
UserCredentialPath: "~/.cloudflared/cert.pem",
}
assert.Equal(t, edgeConnectionConfig, *clientConfig.EdgeConnectionConfig)
dohProxyConfig := DoHProxyConfig{
ListenHost: "localhost",
ListenPort: 53,
Upstreams: []string{"https://1.1.1.1/dns-query", "https://1.0.0.1/dns-query"},
}
assert.Len(t, clientConfig.DoHProxyConfigs, 1)
assert.Equal(t, dohProxyConfig, *clientConfig.DoHProxyConfigs[0])
reverseProxyConfig := ReverseProxyConfig{
TunnelHostname: "sdfjadk33.cftunnel.com",
OriginConfigJSONHandler: &OriginConfigJSONHandler{
OriginConfig: &HTTPOriginConfig{
URLString: "https://127.0.0.1:8080",
TCPKeepAlive: time.Second * 30,
DialDualStack: true,
TLSHandshakeTimeout: time.Second * 10,
TLSVerify: true,
OriginCAPool: "",
OriginServerName: "",
MaxIdleConnections: 100,
IdleConnectionTimeout: time.Second * 90,
ProxyConnectionTimeout: time.Second * 90,
ExpectContinueTimeout: time.Second * 90,
ChunkedEncoding: true,
},
},
Retries: 5,
ConnectionTimeout: 30,
CompressionQuality: 0,
}
assert.Len(t, clientConfig.ReverseProxyConfigs, 1)
assert.Equal(t, reverseProxyConfig, *clientConfig.ReverseProxyConfigs[0])
}
func TestMarshalFallibleConfig(t *testing.T) {
tests := []struct {
fallibleConfig FallibleConfig
expctedJSONLiteral string
}{
{
fallibleConfig: sampleSupervisorConfig(),
expctedJSONLiteral: `{
"supervisor_config":{
"auto_update_frequency":75600000000000,
"metrics_update_frequency":660000000000,
"grace_period":31000000000
}
}`,
},
{
fallibleConfig: sampleEdgeConnectionConfig(),
expctedJSONLiteral: `{
"edge_connection_config":{
"num_ha_connections":49,
"heartbeat_interval":5000000000,
"timeout":9000000000,
"max_failed_heartbeats":9001,
"user_credential_path":"/Users/example/.cloudflared/cert.pem"
}
}`,
},
{
fallibleConfig: sampleDoHProxyConfig(),
expctedJSONLiteral: `{
"doh_proxy_config":{
"listen_host":"127.0.0.1",
"listen_port":53,
"upstreams":["1.1.1.1","1.0.0.1"]
}
}`,
},
{
fallibleConfig: sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{sampleHTTPOriginConfig()}
}),
expctedJSONLiteral: `{
"reverse_proxy_config":{
"tunnel_hostname":"mock-non-lb-tunnel.example.com",
"origin_config":{
"Http":{
"url_string":"https.example.com",
"tcp_keep_alive":7000000000,
"dial_dual_stack":true,
"tls_handshake_timeout":11000000000,
"tls_verify":true,
"origin_ca_pool":"/etc/cert.pem",
"origin_server_name":"secure.example.com",
"max_idle_connections":19,
"idle_connection_timeout":17000000000,
"proxy_connection_timeout":15000000000,
"expect_continue_timeout":21000000000,
"chunked_encoding":true
}
},
"retries":18,
"connection_timeout":5000000000,
"compression_quality":3
}
}`,
},
{
fallibleConfig: sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{sampleWebSocketOriginConfig()}
}),
expctedJSONLiteral: `{
"reverse_proxy_config":{
"tunnel_hostname":"mock-non-lb-tunnel.example.com",
"origin_config":{
"WebSocket":{
"url_string":"ssh://example.com",
"tls_verify":true,
"origin_ca_pool":"/etc/cert.pem",
"origin_server_name":"secure.example.com"
}
},
"retries":18,
"connection_timeout":5000000000,
"compression_quality":3
}
}`,
},
{
fallibleConfig: sampleReverseProxyConfig(func(c *ReverseProxyConfig) {
c.OriginConfigJSONHandler = &OriginConfigJSONHandler{&HelloWorldOriginConfig{}}
}),
expctedJSONLiteral: `{
"reverse_proxy_config":{
"tunnel_hostname":"mock-non-lb-tunnel.example.com",
"origin_config":{
"HelloWorld":{}
},
"retries":18,
"connection_timeout":5000000000,
"compression_quality":3
}
}`,
},
}
for _, test := range tests {
b, err := json.Marshal(test.fallibleConfig)
assert.NoError(t, err)
assert.Equal(t, prettyToValidJSON(test.expctedJSONLiteral), string(b))
}
}
type prettyJSON string
func prettyToValidJSON(prettyJSON string) string {
return strings.ReplaceAll(strings.ReplaceAll(prettyJSON, "\n", ""), "\t", "")
}
+70
View File
@@ -0,0 +1,70 @@
package pogs
import (
"context"
"github.com/cloudflare/cloudflared/tunnelrpc"
"zombiezen.com/go/capnproto2/server"
)
func (i TunnelServer_PogsImpl) ReconnectTunnel(p tunnelrpc.TunnelServer_reconnectTunnel) error {
jwt, err := p.Params.Jwt()
if err != nil {
return err
}
hostname, err := p.Params.Hostname()
if err != nil {
return err
}
options, err := p.Params.Options()
if err != nil {
return err
}
pogsOptions, err := UnmarshalRegistrationOptions(options)
if err != nil {
return err
}
server.Ack(p.Options)
registration, err := i.impl.ReconnectTunnel(p.Ctx, jwt, hostname, pogsOptions)
if err != nil {
return err
}
result, err := p.Results.NewResult()
if err != nil {
return err
}
return MarshalTunnelRegistration(result, registration)
}
func (c TunnelServer_PogsClient) ReconnectTunnel(
ctx context.Context,
jwt []byte,
hostname string,
options *RegistrationOptions,
) (*TunnelRegistration, error) {
client := tunnelrpc.TunnelServer{Client: c.Client}
promise := client.ReconnectTunnel(ctx, func(p tunnelrpc.TunnelServer_reconnectTunnel_Params) error {
err := p.SetJwt(jwt)
if err != nil {
return err
}
err = p.SetHostname(hostname)
if err != nil {
return err
}
registrationOptions, err := p.NewOptions()
if err != nil {
return err
}
err = MarshalRegistrationOptions(registrationOptions, options)
if err != nil {
return err
}
return nil
})
retval, err := promise.Result().Struct()
if err != nil {
return nil, err
}
return UnmarshalTunnelRegistration(retval)
}
+131 -41
View File
@@ -2,10 +2,12 @@ package pogs
import (
"context"
"fmt"
"time"
"github.com/cloudflare/cloudflared/tunnelrpc"
"github.com/google/uuid"
"github.com/pkg/errors"
log "github.com/sirupsen/logrus"
capnp "zombiezen.com/go/capnproto2"
@@ -31,11 +33,12 @@ func UnmarshalAuthentication(s tunnelrpc.Authentication) (*Authentication, error
}
type TunnelRegistration struct {
Err string
Url string
LogLines []string
PermanentFailure bool
TunnelID string `capnp:"tunnelID"`
Err string
Url string
LogLines []string
PermanentFailure bool
TunnelID string `capnp:"tunnelID"`
RetryAfterSeconds uint16
}
func MarshalTunnelRegistration(s tunnelrpc.TunnelRegistration, p *TunnelRegistration) error {
@@ -61,6 +64,7 @@ type RegistrationOptions struct {
RunFromTerminal bool `capnp:"runFromTerminal"`
CompressionQuality uint64 `capnp:"compressionQuality"`
UUID string `capnp:"uuid"`
NumPreviousAttempts uint8
}
func MarshalRegistrationOptions(s tunnelrpc.RegistrationOptions, p *RegistrationOptions) error {
@@ -73,28 +77,129 @@ func UnmarshalRegistrationOptions(s tunnelrpc.RegistrationOptions) (*Registratio
return p, err
}
type ConnectResult struct {
Err *ConnectError
ServerInfo ServerInfo
ClientConfig ClientConfig
// ConnectResult models the result of Connect RPC, implemented by ConnectError and ConnectSuccess.
type ConnectResult interface {
ConnectError() *ConnectError
ConnectedTo() string
ClientConfig() *ClientConfig
Marshal(s tunnelrpc.ConnectResult) error
}
func MarshalConnectResult(s tunnelrpc.ConnectResult, p *ConnectResult) error {
return pogs.Insert(tunnelrpc.ConnectResult_TypeID, s.Struct, p)
func MarshalConnectResult(s tunnelrpc.ConnectResult, p ConnectResult) error {
return p.Marshal(s)
}
func UnmarshalConnectResult(s tunnelrpc.ConnectResult) (*ConnectResult, error) {
p := new(ConnectResult)
err := pogs.Extract(p, tunnelrpc.ConnectResult_TypeID, s.Struct)
return p, err
func UnmarshalConnectResult(s tunnelrpc.ConnectResult) (ConnectResult, error) {
switch s.Result().Which() {
case tunnelrpc.ConnectResult_result_Which_err:
capnpConnectError, err := s.Result().Err()
if err != nil {
return nil, err
}
return UnmarshalConnectError(capnpConnectError)
case tunnelrpc.ConnectResult_result_Which_success:
capnpConnectSuccess, err := s.Result().Success()
if err != nil {
return nil, err
}
return UnmarshalConnectSuccess(capnpConnectSuccess)
default:
return nil, fmt.Errorf("Unmarshal %v not implemented yet", s.Result().Which().String())
}
}
// ConnectSuccess is the concrete returned type when Connect RPC succeed
type ConnectSuccess struct {
ServerLocationName string
Config *ClientConfig
}
func (*ConnectSuccess) ConnectError() *ConnectError {
return nil
}
func (cs *ConnectSuccess) ConnectedTo() string {
return cs.ServerLocationName
}
func (cs *ConnectSuccess) ClientConfig() *ClientConfig {
return cs.Config
}
func (cs *ConnectSuccess) Marshal(s tunnelrpc.ConnectResult) error {
capnpConnectSuccess, err := s.Result().NewSuccess()
if err != nil {
return err
}
err = capnpConnectSuccess.SetServerLocationName(cs.ServerLocationName)
if err != nil {
return errors.Wrap(err, "failed to set ConnectSuccess.ServerLocationName")
}
if cs.Config != nil {
capnpClientConfig, err := capnpConnectSuccess.NewClientConfig()
if err != nil {
return errors.Wrap(err, "failed to initialize ConnectSuccess.ClientConfig")
}
if err := MarshalClientConfig(capnpClientConfig, cs.Config); err != nil {
return errors.Wrap(err, "failed to marshal ClientConfig")
}
}
return nil
}
func UnmarshalConnectSuccess(s tunnelrpc.ConnectSuccess) (*ConnectSuccess, error) {
p := new(ConnectSuccess)
serverLocationName, err := s.ServerLocationName()
if err != nil {
return nil, errors.Wrap(err, "failed to get tunnelrpc.ConnectSuccess.ServerLocationName")
}
p.ServerLocationName = serverLocationName
if s.HasClientConfig() {
capnpClientConfig, err := s.ClientConfig()
if err != nil {
return nil, errors.Wrap(err, "failed to get tunnelrpc.ConnectSuccess.ClientConfig")
}
p.Config, err = UnmarshalClientConfig(capnpClientConfig)
if err != nil {
return nil, errors.Wrap(err, "failed to get unmarshal ClientConfig")
}
}
return p, nil
}
// ConnectError is the concrete returned type when Connect RPC encounters some error
type ConnectError struct {
Cause string
RetryAfter time.Duration
ShouldRetry bool
}
func (ce *ConnectError) ConnectError() *ConnectError {
return ce
}
func (*ConnectError) ConnectedTo() string {
return ""
}
func (*ConnectError) ClientConfig() *ClientConfig {
return nil
}
func (ce *ConnectError) Marshal(s tunnelrpc.ConnectResult) error {
capnpConnectError, err := s.Result().NewErr()
if err != nil {
return err
}
return MarshalConnectError(capnpConnectError, ce)
}
func MarshalConnectError(s tunnelrpc.ConnectError, p *ConnectError) error {
return pogs.Insert(tunnelrpc.ConnectError_TypeID, s.Struct, p)
}
@@ -134,8 +239,7 @@ type ConnectParameters struct {
NumPreviousAttempts uint8
Tags []Tag
CloudflaredVersion string
Name string
Group string
IntentLabel string
}
func MarshalConnectParameters(s tunnelrpc.CapnpConnectParameters, p *ConnectParameters) error {
@@ -168,13 +272,7 @@ func MarshalConnectParameters(s tunnelrpc.CapnpConnectParameters, p *ConnectPara
if err := s.SetCloudflaredVersion(p.CloudflaredVersion); err != nil {
return err
}
if err := s.SetName(p.Name); err != nil {
return err
}
if err := s.SetGroup(p.Group); err != nil {
return err
}
return nil
return s.SetIntentLabel(p.IntentLabel)
}
func UnmarshalConnectParameters(s tunnelrpc.CapnpConnectParameters) (*ConnectParameters, error) {
@@ -215,24 +313,14 @@ func UnmarshalConnectParameters(s tunnelrpc.CapnpConnectParameters) (*ConnectPar
return nil, err
}
name, err := s.Name()
if err != nil {
return nil, err
}
group, err := s.Group()
if err != nil {
return nil, err
}
intentLabel, err := s.IntentLabel()
return &ConnectParameters{
OriginCert: originCert,
CloudflaredID: cloudflaredID,
NumPreviousAttempts: s.NumPreviousAttempts(),
Tags: tags,
CloudflaredVersion: cloudflaredVersion,
Name: name,
Group: group,
IntentLabel: intentLabel,
}, nil
}
@@ -240,7 +328,9 @@ type TunnelServer interface {
RegisterTunnel(ctx context.Context, originCert []byte, hostname string, options *RegistrationOptions) (*TunnelRegistration, error)
GetServerInfo(ctx context.Context) (*ServerInfo, error)
UnregisterTunnel(ctx context.Context, gracePeriodNanoSec int64) error
Connect(ctx context.Context, paramaters *ConnectParameters) (*ConnectResult, error)
Connect(ctx context.Context, parameters *ConnectParameters) (ConnectResult, error)
Authenticate(ctx context.Context, originCert []byte, hostname string, options *RegistrationOptions) (*AuthenticateResponse, error)
ReconnectTunnel(ctx context.Context, jwt []byte, hostname string, options *RegistrationOptions) (*TunnelRegistration, error)
}
func TunnelServer_ServerToClient(s TunnelServer) tunnelrpc.TunnelServer {
@@ -301,11 +391,11 @@ func (i TunnelServer_PogsImpl) UnregisterTunnel(p tunnelrpc.TunnelServer_unregis
}
func (i TunnelServer_PogsImpl) Connect(p tunnelrpc.TunnelServer_connect) error {
paramaters, err := p.Params.Parameters()
parameters, err := p.Params.Parameters()
if err != nil {
return err
}
pogsParameters, err := UnmarshalConnectParameters(paramaters)
pogsParameters, err := UnmarshalConnectParameters(parameters)
if err != nil {
return err
}
@@ -318,7 +408,7 @@ func (i TunnelServer_PogsImpl) Connect(p tunnelrpc.TunnelServer_connect) error {
if err != nil {
return err
}
return MarshalConnectResult(result, connectResult)
return connectResult.Marshal(result)
}
type TunnelServer_PogsClient struct {
@@ -382,7 +472,7 @@ func (c TunnelServer_PogsClient) UnregisterTunnel(ctx context.Context, gracePeri
func (c TunnelServer_PogsClient) Connect(ctx context.Context,
parameters *ConnectParameters,
) (*ConnectResult, error) {
) (ConnectResult, error) {
client := tunnelrpc.TunnelServer{Client: c.Client}
promise := client.Connect(ctx, func(p tunnelrpc.TunnelServer_connect_Params) error {
connectParameters, err := p.NewParameters()
+43 -14
View File
@@ -11,21 +11,51 @@ import (
capnp "zombiezen.com/go/capnproto2"
)
func sampleTestConnectResult() *ConnectResult {
return &ConnectResult{
Err: &ConnectError{
func TestTunnelRegistration(t *testing.T) {
testCases := []*TunnelRegistration{
&TunnelRegistration{
Err: "it broke",
Url: "asdf.cftunnel.com",
LogLines: []string{"it", "was", "broken"},
PermanentFailure: true,
TunnelID: "asdfghjkl;",
RetryAfterSeconds: 19,
},
}
for i, testCase := range testCases {
_, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))
capnpEntity, err := tunnelrpc.NewTunnelRegistration(seg)
if !assert.NoError(t, err) {
t.Fatal("Couldn't initialize a new message")
}
err = MarshalTunnelRegistration(capnpEntity, testCase)
if !assert.NoError(t, err, "testCase #%v failed to marshal", i) {
continue
}
result, err := UnmarshalTunnelRegistration(capnpEntity)
if !assert.NoError(t, err, "testCase #%v failed to unmarshal", i) {
continue
}
assert.Equal(t, testCase, result, "testCase index %v didn't preserve struct through marshalling and unmarshalling", i)
}
}
func TestConnectResult(t *testing.T) {
testCases := []ConnectResult{
&ConnectError{
Cause: "it broke",
ShouldRetry: false,
RetryAfter: 2 * time.Second,
},
ServerInfo: ServerInfo{LocationName: "computer"},
ClientConfig: *sampleClientConfig(),
}
}
func TestConnectResult(t *testing.T) {
testCases := []*ConnectResult{
sampleTestConnectResult(),
&ConnectSuccess{
ServerLocationName: "SFO",
Config: sampleClientConfig(),
},
&ConnectSuccess{
ServerLocationName: "",
Config: nil,
},
}
for i, testCase := range testCases {
_, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))
@@ -49,7 +79,7 @@ func TestConnectParameters(t *testing.T) {
testCases := []*ConnectParameters{
sampleConnectParameters(),
sampleConnectParameters(func(c *ConnectParameters) {
c.Name = ""
c.IntentLabel = "my_intent"
}),
sampleConnectParameters(func(c *ConnectParameters) {
c.Tags = nil
@@ -89,8 +119,7 @@ func sampleConnectParameters(overrides ...func(*ConnectParameters)) *ConnectPara
},
},
CloudflaredVersion: "7.0",
Name: "My Computer",
Group: "www",
IntentLabel: "my_intent",
}
sample.ensureNoZeroFields()
for _, f := range overrides {
+27 -10
View File
@@ -19,6 +19,8 @@ struct TunnelRegistration {
permanentFailure @3 :Bool;
# Displayed to user
tunnelID @4 :Text;
# How long should this connection wait to retry in seconds, if the error wasn't permanent
retryAfterSeconds @5 :UInt16;
}
struct RegistrationOptions {
@@ -44,6 +46,8 @@ struct RegistrationOptions {
# cross stream compression setting, 0 - off, 3 - high
compressionQuality @10 :UInt64;
uuid @11 :Text;
# number of previous attempts to send RegisterTunnel/ReconnectTunnel
numPreviousAttempts @12 :UInt8;
}
struct CapnpConnectParameters {
@@ -57,18 +61,15 @@ struct CapnpConnectParameters {
tags @3 :List(Tag);
# release version of cloudflared
cloudflaredVersion @4 :Text;
# friendly name for this cloudflared instance
name @5 :Text;
# group whose behavior this cloudflared instance will adopt
group @6 :Text;
# which intent this cloudflared instance should get its behaviour from
intentLabel @5 :Text;
}
struct ConnectResult {
err @0 :ConnectError;
# Information about the server this connection is established with
serverInfo @1 :ServerInfo;
# How this cloudflared instance should be configured
clientConfig @2 :ClientConfig;
result :union {
err @0 :ConnectError;
success @1 :ConnectSuccess;
}
}
struct ConnectError {
@@ -78,6 +79,13 @@ struct ConnectError {
shouldRetry @2 :Bool;
}
struct ConnectSuccess {
# Information about the server this connection is established with
serverLocationName @0 :Text;
# How this cloudflared instance should be configured. This can be null if there isn't an intent for this origin yet
clientConfig @1 :ClientConfig;
}
struct ClientConfig {
# Version of this configuration. This value is opaque, but is guaranteed
# to monotonically increase in value. Any configuration supplied to
@@ -129,7 +137,7 @@ struct EdgeConnectionConfig {
struct ReverseProxyConfig {
tunnelHostname @0 :Text;
origin :union {
originConfig :union {
http @1 :HTTPOriginConfig;
websocket @2 :WebSocketOriginConfig;
helloWorld @3 :HelloWorldOriginConfig;
@@ -270,11 +278,20 @@ struct FailedConfig {
reason @4 :Text;
}
struct AuthenticateResponse {
permanentErr @0 :Text;
retryableErr @1 :Text;
jwt @2 :Data;
hoursUntilRefresh @3 :UInt8;
}
interface TunnelServer {
registerTunnel @0 (originCert :Data, hostname :Text, options :RegistrationOptions) -> (result :TunnelRegistration);
getServerInfo @1 () -> (result :ServerInfo);
unregisterTunnel @2 (gracePeriodNanoSec :Int64) -> ();
connect @3 (parameters :CapnpConnectParameters) -> (result :ConnectResult);
authenticate @4 (originCert :Data, hostname :Text, options :RegistrationOptions) -> (result :AuthenticateResponse);
reconnectTunnel @5 (jwt :Data, hostname :Text, options :RegistrationOptions) -> (result :TunnelRegistration);
}
interface ClientService {
+1090 -331
View File
File diff suppressed because it is too large Load Diff
+64 -2
View File
@@ -1,6 +1,7 @@
package validation
import (
"context"
"fmt"
"net"
"net/url"
@@ -11,13 +12,19 @@ import (
"github.com/pkg/errors"
"golang.org/x/net/idna"
"gopkg.in/coreos/go-oidc.v2"
)
const defaultScheme = "http"
const (
defaultScheme = "http"
accessDomain = "cloudflareaccess.com"
accessCertPath = "/cdn-cgi/access/certs"
accessJwtHeader = "Cf-access-jwt-assertion"
)
var (
supportedProtocols = []string{"http", "https", "rdp"}
validationTimeout = time.Duration(30 * time.Second)
validationTimeout = time.Duration(30 * time.Second)
)
func ValidateHostname(hostname string) (string, error) {
@@ -53,6 +60,12 @@ func ValidateHostname(hostname string) (string, error) {
}
// ValidateUrl returns a validated version of `originUrl` with a scheme prepended (by default http://).
// Note: when originUrl contains a scheme, the path is removed:
// ValidateUrl("https://localhost:8080/api/") => "https://localhost:8080"
// but when it does not, the path is preserved:
// ValidateUrl("localhost:8080/api/") => "http://localhost:8080/api/"
// This is arguably a bug, but changing it might break some cloudflared users.
func ValidateUrl(originUrl string) (string, error) {
if originUrl == "" {
return "", fmt.Errorf("URL should not be empty")
@@ -114,6 +127,8 @@ func ValidateUrl(originUrl string) (string, error) {
if err != nil {
return "", fmt.Errorf("URL %s has invalid format", originUrl)
}
// This is why the path is preserved when `originUrl` doesn't have a schema.
// Using `parsedUrl.Port()` here, instead of `port`, would remove the path
return fmt.Sprintf("%s://%s", defaultScheme, net.JoinHostPort(hostname, port)), nil
}
}
@@ -197,3 +212,50 @@ func toggleProtocol(httpProtocol string) string {
return httpProtocol
}
}
// Access checks if a JWT from Cloudflare Access is valid.
type Access struct {
verifier *oidc.IDTokenVerifier
}
func NewAccessValidator(ctx context.Context, domain, issuer, applicationAUD string) (*Access, error) {
domainURL, err := ValidateUrl(domain)
if err != nil {
return nil, err
}
issuerURL, err := ValidateUrl(issuer)
if err != nil {
return nil, err
}
// An issuerURL from Cloudflare Access will always use HTTPS.
issuerURL = strings.Replace(issuerURL, "http:", "https:", 1)
keySet := oidc.NewRemoteKeySet(ctx, domainURL+accessCertPath)
return &Access{oidc.NewVerifier(issuerURL, keySet, &oidc.Config{ClientID: applicationAUD})}, nil
}
func (a *Access) Validate(ctx context.Context, jwt string) error {
token, err := a.verifier.Verify(ctx, jwt)
if err != nil {
return errors.Wrapf(err, "token is invalid: %s", jwt)
}
// Perform extra sanity checks, just to be safe.
if token == nil {
return fmt.Errorf("token is nil: %s", jwt)
}
if !strings.HasSuffix(token.Issuer, accessDomain) {
return fmt.Errorf("token has non-cloudflare issuer of %s: %s", token.Issuer, jwt)
}
return nil
}
func (a *Access) ValidateRequest(ctx context.Context, r *http.Request) error {
return a.Validate(ctx, r.Header.Get(accessJwtHeader))
}
+84 -84
View File
@@ -53,98 +53,65 @@ func TestValidateHostname(t *testing.T) {
}
func TestValidateUrl(t *testing.T) {
type testCase struct {
input string
expectedOutput string
}
testCases := []testCase{
{"http://localhost", "http://localhost"},
{"http://localhost/", "http://localhost"},
{"http://localhost/api", "http://localhost"},
{"http://localhost/api/", "http://localhost"},
{"https://localhost", "https://localhost"},
{"https://localhost/", "https://localhost"},
{"https://localhost/api", "https://localhost"},
{"https://localhost/api/", "https://localhost"},
{"https://localhost:8080", "https://localhost:8080"},
{"https://localhost:8080/", "https://localhost:8080"},
{"https://localhost:8080/api", "https://localhost:8080"},
{"https://localhost:8080/api/", "https://localhost:8080"},
{"localhost", "http://localhost"},
{"localhost/", "http://localhost/"},
{"localhost/api", "http://localhost/api"},
{"localhost/api/", "http://localhost/api/"},
{"localhost:8080", "http://localhost:8080"},
{"localhost:8080/", "http://localhost:8080/"},
{"localhost:8080/api", "http://localhost:8080/api"},
{"localhost:8080/api/", "http://localhost:8080/api/"},
{"localhost:8080/api/?asdf", "http://localhost:8080/api/?asdf"},
{"http://127.0.0.1:8080", "http://127.0.0.1:8080"},
{"127.0.0.1:8080", "http://127.0.0.1:8080"},
{"127.0.0.1", "http://127.0.0.1"},
{"https://127.0.0.1:8080", "https://127.0.0.1:8080"},
{"[::1]:8080", "http://[::1]:8080"},
{"http://[::1]", "http://[::1]"},
{"http://[::1]:8080", "http://[::1]:8080"},
{"[::1]", "http://[::1]"},
{"https://example.com", "https://example.com"},
{"example.com", "http://example.com"},
{"http://hello.example.com", "http://hello.example.com"},
{"hello.example.com", "http://hello.example.com"},
{"hello.example.com:8080", "http://hello.example.com:8080"},
{"https://hello.example.com:8080", "https://hello.example.com:8080"},
{"https://bücher.example.com", "https://xn--bcher-kva.example.com"},
{"bücher.example.com", "http://xn--bcher-kva.example.com"},
{"https%3A%2F%2Fhello.example.com", "https://hello.example.com"},
{"https://alex:12345@hello.example.com:8080", "https://hello.example.com:8080"},
}
for i, testCase := range testCases {
validUrl, err := ValidateUrl(testCase.input)
assert.NoError(t, err, "test case %v", i)
assert.Equal(t, testCase.expectedOutput, validUrl, "test case %v", i)
}
validUrl, err := ValidateUrl("")
assert.Equal(t, fmt.Errorf("URL should not be empty"), err)
assert.Empty(t, validUrl)
validUrl, err = ValidateUrl("https://localhost:8080")
assert.Nil(t, err)
assert.Equal(t, "https://localhost:8080", validUrl)
validUrl, err = ValidateUrl("localhost:8080")
assert.Nil(t, err)
assert.Equal(t, "http://localhost:8080", validUrl)
validUrl, err = ValidateUrl("http://localhost")
assert.Nil(t, err)
assert.Equal(t, "http://localhost", validUrl)
validUrl, err = ValidateUrl("http://127.0.0.1:8080")
assert.Nil(t, err)
assert.Equal(t, "http://127.0.0.1:8080", validUrl)
validUrl, err = ValidateUrl("127.0.0.1:8080")
assert.Nil(t, err)
assert.Equal(t, "http://127.0.0.1:8080", validUrl)
validUrl, err = ValidateUrl("127.0.0.1")
assert.Nil(t, err)
assert.Equal(t, "http://127.0.0.1", validUrl)
validUrl, err = ValidateUrl("https://127.0.0.1:8080")
assert.Nil(t, err)
assert.Equal(t, "https://127.0.0.1:8080", validUrl)
validUrl, err = ValidateUrl("[::1]:8080")
assert.Nil(t, err)
assert.Equal(t, "http://[::1]:8080", validUrl)
validUrl, err = ValidateUrl("http://[::1]")
assert.Nil(t, err)
assert.Equal(t, "http://[::1]", validUrl)
validUrl, err = ValidateUrl("http://[::1]:8080")
assert.Nil(t, err)
assert.Equal(t, "http://[::1]:8080", validUrl)
validUrl, err = ValidateUrl("[::1]")
assert.Nil(t, err)
assert.Equal(t, "http://[::1]", validUrl)
validUrl, err = ValidateUrl("https://example.com")
assert.Nil(t, err)
assert.Equal(t, "https://example.com", validUrl)
validUrl, err = ValidateUrl("example.com")
assert.Nil(t, err)
assert.Equal(t, "http://example.com", validUrl)
validUrl, err = ValidateUrl("http://hello.example.com")
assert.Nil(t, err)
assert.Equal(t, "http://hello.example.com", validUrl)
validUrl, err = ValidateUrl("hello.example.com")
assert.Nil(t, err)
assert.Equal(t, "http://hello.example.com", validUrl)
validUrl, err = ValidateUrl("hello.example.com:8080")
assert.Nil(t, err)
assert.Equal(t, "http://hello.example.com:8080", validUrl)
validUrl, err = ValidateUrl("https://hello.example.com:8080")
assert.Nil(t, err)
assert.Equal(t, "https://hello.example.com:8080", validUrl)
validUrl, err = ValidateUrl("https://bücher.example.com")
assert.Nil(t, err)
assert.Equal(t, "https://xn--bcher-kva.example.com", validUrl)
validUrl, err = ValidateUrl("bücher.example.com")
assert.Nil(t, err)
assert.Equal(t, "http://xn--bcher-kva.example.com", validUrl)
validUrl, err = ValidateUrl("https%3A%2F%2Fhello.example.com")
assert.Nil(t, err)
assert.Equal(t, "https://hello.example.com", validUrl)
validUrl, err = ValidateUrl("ftp://alex:12345@hello.example.com:8080/robot.txt")
assert.Equal(t, "Currently Argo Tunnel does not support ftp protocol.", err.Error())
assert.Empty(t, validUrl)
validUrl, err = ValidateUrl("https://alex:12345@hello.example.com:8080")
assert.Nil(t, err)
assert.Equal(t, "https://hello.example.com:8080", validUrl)
}
func TestToggleProtocol(t *testing.T) {
@@ -320,6 +287,39 @@ func TestValidateHTTPService_NonResponsiveOrigin(t *testing.T) {
}
}
func TestNewAccessValidatorOk(t *testing.T) {
ctx := context.Background()
url := "test.cloudflareaccess.com"
access, err := NewAccessValidator(ctx, url, url, "")
assert.NoError(t, err)
assert.NotNil(t, access)
assert.Error(t, access.Validate(ctx, ""))
assert.Error(t, access.Validate(ctx, "invalid"))
req := httptest.NewRequest("GET", "https://test.cloudflareaccess.com", nil)
req.Header.Set(accessJwtHeader, "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c")
assert.Error(t, access.ValidateRequest(ctx, req))
}
func TestNewAccessValidatorErr(t *testing.T) {
ctx := context.Background()
urls := []string{
"",
"tcp://test.cloudflareaccess.com",
"wss://cloudflarenone.com",
}
for _, url := range urls {
access, err := NewAccessValidator(ctx, url, url, "")
assert.Error(t, err, url)
assert.Nil(t, access)
}
}
type testRoundTripper func(req *http.Request) (*http.Response, error)
func (f testRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Some files were not shown because too many files have changed in this diff Show More