Compare commits

...

21 Commits

Author SHA1 Message Date
Devin Carr 7ecb6d3e88 Release 2023.4.2 2023-04-24 12:48:58 -07:00
Devin Carr 88c25d2c67 TUN-7133: Add sampling support for streaming logs
In addition to supporting sampling support for streaming logs,
cloudflared tail also supports this via `--sample 0.5` to sample 50%
of your log events.
2023-04-24 09:39:26 -07:00
Devin Carr 38cd455e4d TUN-7373: Streaming logs override for same actor
To help accommodate web browser interactions with websockets, when a
streaming logs session is requested for the same actor while already
serving a session for that user in a separate request, the original
request will be closed and the new request start streaming logs
instead. This should help with rogue sessions holding on for too long
with no client on the other side (before idle timeout or connection
close).
2023-04-21 11:54:37 -07:00
Devin Carr ee5e447d44 TUN-7141: Add component tests for streaming logs 2023-04-21 10:14:03 -07:00
Sudarsan Reddy 4d30a71434 TUN-7383: Bump requirements.txt 2023-04-20 16:49:26 +01:00
Jesse Li 39b7aed24e AUTH-4887 Add aud parameter to token transfer url 2023-04-19 21:01:24 +00:00
Devin Carr 4de1bc4bba TUN-7378: Remove RPC debug logs 2023-04-19 18:35:51 +00:00
Sudarsan Reddy e426693330 TUN-7361: Add a label to override hostname
It might make sense for users to sometimes name their cloudflared
connectors to make identification easier than relying on hostnames that
TUN-7360 provides. This PR provides a new --label option to cloudflared
tunnel that a user could provide to give custom names to their
connectors.
2023-04-19 13:56:32 +00:00
Devin Carr 0b5b9b8297 TUN-7130: Categorize UDP logs for streaming logs 2023-04-18 20:49:36 +00:00
Devin Carr 7a014b06ec TUN-7129: Categorize TCP logs for streaming logs 2023-04-18 20:49:29 +00:00
James Royal 171d4ac77c AUTH-3122 Verify that Access tokens are still valid in curl command
Before this change, the only sure fire way to make sure you had a valid
Access token was to run `cloudflared access login <your domain>`. That
was because that command would actually make a preflight request to ensure
that the edge considered that token valid. The most common reasons a token
was no longer valid was expiration and revocation. Expiration is easy to
check client side, but revocation can only be checked at the edge.

This change adds the same flow that cfd access login did to the curl command.
It will preflight the request with the token and ensure that the edge thinks
its valid before making the real request.
2023-04-18 13:38:50 +00:00
Sudarsan Reddy 5e212a6bf3 TUN-7360: Add Get Host Details handler in management service
With the management tunnels work, we allow calls to our edge service
   using an access JWT provided by Tunnelstore. Given a connector ID,
   this request is then proxied to the appropriate Cloudflare Tunnel.

   This PR takes advantage of this flow and adds a new host_details
   endpoint. Calls to this endpoint will result in cloudflared gathering
   some details about the host: hostname (os.hostname()) and ip address
   (localAddr in a dial).

   Note that the mini spec lists 4 alternatives and this picks alternative
   3 because:

   1. Ease of implementation: This is quick and non-intrusive to any of our
      code path. We expect to change how connection tracking works and
      regardless of the direction we take, it may be easy to keep, morph
      or throw this away.

   2. The cloudflared part of this round trip takes some time with a
      hostname call and a dial. But note that this is off the critical path
      and not an API that will be exercised often.
2023-04-18 09:54:54 +00:00
Devin Carr 3996b1adca Release 2023.4.1 2023-04-17 10:04:12 -07:00
Devin Carr 71997be90e TUN-7368: Report destination address for TCP requests in logs 2023-04-13 16:49:42 -07:00
Devin Carr 991f01fe34 TUN-7131: Add cloudflared log event to connection messages and enable streaming logs 2023-04-12 14:41:11 -07:00
Devin Carr b89c092c1b TUN-7134: Acquire token for cloudflared tail
cloudflared tail will now fetch the management token from by making
a request to the Cloudflare API using the cert.pem (acquired from
cloudflared login).

Refactored some of the credentials code into it's own package as
to allow for easier use between subcommands outside of
`cloudflared tunnel`.
2023-04-12 09:43:38 -07:00
Devin Carr 8dc0697a8f TUN-7132 TUN-7136: Add filter support for streaming logs
Additionally adds similar support in cloudflared tail to provide
filters for events and log level.
2023-04-11 20:20:52 +00:00
Sudarsan Reddy 5dbf76a7aa TUN-7335: Fix cloudflared update not working in windows
This PR fixes some long standing bugs in the windows update
paths. We previously did not surface the errors at all leading to
this function failing silently.

This PR:

1. Now returns the ExitError if the bat run for update fails.
2. Fixes the errors surfaced by that return:
    a. The batch file doesnt play well with spaces. This is fixed by
    using PROGRA~1/2 which are aliases windows uses.
    b. The existing script also seemed to be irregular about where batch
    files were put and looked for. This is also fixed in this script.
2023-04-11 08:54:38 +00:00
Devin Carr 8d87d4facd TUN-7351: Add streaming logs session ping and timeout
Sends a ping every 15 seconds to keep the session alive even if no
protocol messages are being propagated. Additionally, sets a hard
timeout of 5 minutes when not actively streaming logs to drop the
connection.
2023-04-10 22:14:58 +00:00
Devin Carr 3fd571063e TUN-7128: Categorize logs from public hostname locations
Updates the HTTP ingress request log events to have more structured
fields to adapt to streaming logs reporting.
2023-04-10 22:14:12 +00:00
Devin Carr 5d0bb25572 TUN-7354: Don't warn for empty ingress rules when using --token 2023-04-10 22:12:40 +00:00
163 changed files with 14273 additions and 2122 deletions
+6 -2
View File
@@ -1,6 +1,10 @@
## 2023.4.1
### New Features
- You can now stream your logs from your remote cloudflared to your local terminal with `cloudflared tail <TUNNEL-ID>`. This new feature requires the remote cloudflared to be version 2023.4.1 or higher.
## 2023.3.2
### Notices
- Due to the nature of QuickTunnels (https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/do-more-with-tunnels/trycloudflare/) and its intended usage for testing and experiment of Cloudflare Tunnels, starting from 2023.3.2, QuickTunnels only make a single connection to the edge. If users want to use Tunnels in a production enverionment, they should move to Named Tunnels instead. (https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide/remote/#set-up-a-tunnel-remotely-dashboard-setup)
- Due to the nature of QuickTunnels (https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/do-more-with-tunnels/trycloudflare/) and its intended usage for testing and experiment of Cloudflare Tunnels, starting from 2023.3.2, QuickTunnels only make a single connection to the edge. If users want to use Tunnels in a production environment, they should move to Named Tunnels instead. (https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide/remote/#set-up-a-tunnel-remotely-dashboard-setup)
## 2023.3.1
### Breaking Change
@@ -25,7 +29,7 @@
## 2022.12.0
### Improvements
- cloudflared now attempts to try other edge addresses before falling back to a lower protoocol.
- cloudflared now attempts to try other edge addresses before falling back to a lower protocol.
- cloudflared tunnel no longer spins up a quick tunnel. The call has to be explicit and provide a --url flag.
- cloudflared will now randomly pick the first or second region to connect to instead of always connecting to region2 first.
+23
View File
@@ -1,3 +1,26 @@
2023.4.2
- 2023-04-24 TUN-7133: Add sampling support for streaming logs
- 2023-04-21 TUN-7141: Add component tests for streaming logs
- 2023-04-21 TUN-7373: Streaming logs override for same actor
- 2023-04-20 TUN-7383: Bump requirements.txt
- 2023-04-19 TUN-7361: Add a label to override hostname
- 2023-04-19 TUN-7378: Remove RPC debug logs
- 2023-04-18 TUN-7360: Add Get Host Details handler in management service
- 2023-04-17 AUTH-3122 Verify that Access tokens are still valid in curl command
- 2023-04-17 TUN-7129: Categorize TCP logs for streaming logs
- 2023-04-17 TUN-7130: Categorize UDP logs for streaming logs
- 2023-04-10 AUTH-4887 Add aud parameter to token transfer url
2023.4.1
- 2023-04-13 TUN-7368: Report destination address for TCP requests in logs
- 2023-04-12 TUN-7134: Acquire token for cloudflared tail
- 2023-04-12 TUN-7131: Add cloudflared log event to connection messages and enable streaming logs
- 2023-04-11 TUN-7132 TUN-7136: Add filter support for streaming logs
- 2023-04-06 TUN-7354: Don't warn for empty ingress rules when using --token
- 2023-04-06 TUN-7128: Categorize logs from public hostname locations
- 2023-04-06 TUN-7351: Add streaming logs session ping and timeout
- 2023-04-06 TUN-7335: Fix cloudflared update not working in windows
2023.4.0
- 2023-04-07 TUN-7356: Bump golang.org/x/net package to 0.7.0
- 2023-04-07 TUN-7357: Bump to go 1.19.6
-58
View File
@@ -1,58 +0,0 @@
package certutil
import (
"encoding/json"
"encoding/pem"
"fmt"
)
type namedTunnelToken struct {
ZoneID string `json:"zoneID"`
AccountID string `json:"accountID"`
APIToken string `json:"apiToken"`
}
type OriginCert struct {
ZoneID string
APIToken string
AccountID string
}
func DecodeOriginCert(blocks []byte) (*OriginCert, error) {
if len(blocks) == 0 {
return nil, fmt.Errorf("Cannot decode empty certificate")
}
originCert := OriginCert{}
block, rest := pem.Decode(blocks)
for {
if block == nil {
break
}
switch block.Type {
case "PRIVATE KEY", "CERTIFICATE":
// this is for legacy purposes.
break
case "ARGO TUNNEL TOKEN":
if originCert.ZoneID != "" || originCert.APIToken != "" {
return nil, fmt.Errorf("Found multiple tokens in the certificate")
}
// The token is a string,
// Try the newer JSON format
ntt := namedTunnelToken{}
if err := json.Unmarshal(block.Bytes, &ntt); err == nil {
originCert.ZoneID = ntt.ZoneID
originCert.APIToken = ntt.APIToken
originCert.AccountID = ntt.AccountID
}
default:
return nil, fmt.Errorf("Unknown block %s in the certificate", block.Type)
}
block, rest = pem.Decode(rest)
}
if originCert.ZoneID == "" || originCert.APIToken == "" {
return nil, fmt.Errorf("Missing token in the certificate")
}
return &originCert, nil
}
-51
View File
@@ -1,51 +0,0 @@
package certutil
import (
"fmt"
"io/ioutil"
"testing"
"github.com/stretchr/testify/assert"
)
func TestLoadOriginCert(t *testing.T) {
cert, err := DecodeOriginCert([]byte{})
assert.Equal(t, fmt.Errorf("Cannot decode empty certificate"), err)
assert.Nil(t, cert)
blocks, err := ioutil.ReadFile("test-cert-unknown-block.pem")
assert.Nil(t, err)
cert, err = DecodeOriginCert(blocks)
assert.Equal(t, fmt.Errorf("Unknown block RSA PRIVATE KEY in the certificate"), err)
assert.Nil(t, cert)
}
func TestJSONArgoTunnelTokenEmpty(t *testing.T) {
cert, err := DecodeOriginCert([]byte{})
blocks, err := ioutil.ReadFile("test-cert-no-token.pem")
assert.Nil(t, err)
cert, err = DecodeOriginCert(blocks)
assert.Equal(t, fmt.Errorf("Missing token in the certificate"), err)
assert.Nil(t, cert)
}
func TestJSONArgoTunnelToken(t *testing.T) {
// The given cert's Argo Tunnel Token was generated by base64 encoding this JSON:
// {
// "zoneID": "7b0a4d77dfb881c1a3b7d61ea9443e19",
// "apiToken": "test-service-key",
// "accountID": "abcdabcdabcdabcd1234567890abcdef"
// }
CloudflareTunnelTokenTest(t, "test-cloudflare-tunnel-cert-json.pem")
}
func CloudflareTunnelTokenTest(t *testing.T, path string) {
blocks, err := ioutil.ReadFile(path)
assert.Nil(t, err)
cert, err := DecodeOriginCert(blocks)
assert.Nil(t, err)
assert.NotNil(t, cert)
assert.Equal(t, "7b0a4d77dfb881c1a3b7d61ea9443e19", cert.ZoneID)
key := "test-service-key"
assert.Equal(t, key, cert.APIToken)
}
+1
View File
@@ -8,6 +8,7 @@ type TunnelClient interface {
CreateTunnel(name string, tunnelSecret []byte) (*TunnelWithToken, error)
GetTunnel(tunnelID uuid.UUID) (*Tunnel, error)
GetTunnelToken(tunnelID uuid.UUID) (string, error)
GetManagementToken(tunnelID uuid.UUID) (string, error)
DeleteTunnel(tunnelID uuid.UUID) error
ListTunnels(filter *TunnelFilter) ([]*Tunnel, error)
ListActiveClients(tunnelID uuid.UUID) ([]*ActiveClient, error)
+26
View File
@@ -50,6 +50,10 @@ type newTunnel struct {
TunnelSecret []byte `json:"tunnel_secret"`
}
type managementRequest struct {
Resources []string `json:"resources"`
}
type CleanupParams struct {
queryParams url.Values
}
@@ -133,6 +137,28 @@ func (r *RESTClient) GetTunnelToken(tunnelID uuid.UUID) (token string, err error
return "", r.statusCodeToError("get tunnel token", resp)
}
func (r *RESTClient) GetManagementToken(tunnelID uuid.UUID) (token string, err error) {
endpoint := r.baseEndpoints.accountLevel
endpoint.Path = path.Join(endpoint.Path, fmt.Sprintf("%v/management", tunnelID))
body := &managementRequest{
Resources: []string{"logs"},
}
resp, err := r.sendRequest("POST", endpoint, body)
if err != nil {
return "", errors.Wrap(err, "REST request failed")
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusOK {
err = parseResponse(resp.Body, &token)
return token, err
}
return "", r.statusCodeToError("get tunnel token", resp)
}
func (r *RESTClient) DeleteTunnel(tunnelID uuid.UUID) error {
endpoint := r.baseEndpoints.accountLevel
endpoint.Path = path.Join(endpoint.Path, fmt.Sprintf("%v", tunnelID))
+7
View File
@@ -280,6 +280,13 @@ func curl(c *cli.Context) error {
if err != nil {
return err
}
// Verify that the existing token is still good; if not fetch a new one
if err := verifyTokenAtEdge(appURL, appInfo, c, log); err != nil {
log.Err(err).Msg("Could not verify token")
return err
}
tok, err := token.GetAppTokenIfExists(appInfo)
if err != nil || tok == "" {
if allowRequest {
+4
View File
@@ -47,3 +47,7 @@ func (bi *BuildInfo) GetBuildTypeMsg() string {
}
return fmt.Sprintf(" with %s", bi.BuildType)
}
func (bi *BuildInfo) UserAgent() string {
return fmt.Sprintf("cloudflared/%s", bi.CloudflaredVersion)
}
+1 -1
View File
@@ -90,7 +90,7 @@ func main() {
updater.Init(Version)
tracing.Init(Version)
token.Init(Version)
tail.Init(Version)
tail.Init(bInfo)
runApp(app, graceShutdownC)
}
+188 -12
View File
@@ -2,6 +2,7 @@ package tail
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
@@ -10,28 +11,64 @@ import (
"syscall"
"time"
"github.com/google/uuid"
"github.com/mattn/go-colorable"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"nhooyr.io/websocket"
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/credentials"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/management"
)
var (
version string
buildInfo *cliutil.BuildInfo
)
func Init(v string) {
version = v
func Init(bi *cliutil.BuildInfo) {
buildInfo = bi
}
func Command() *cli.Command {
subcommands := []*cli.Command{
buildTailManagementTokenSubcommand(),
}
return buildTailCommand(subcommands)
}
func buildTailManagementTokenSubcommand() *cli.Command {
return &cli.Command{
Name: "tail",
Action: Run,
Usage: "Stream logs from a remote cloudflared",
Name: "token",
Action: cliutil.ConfiguredAction(managementTokenCommand),
Usage: "Get management access jwt",
UsageText: "cloudflared tail token TUNNEL_ID",
Description: `Get management access jwt for a tunnel`,
Hidden: true,
}
}
func managementTokenCommand(c *cli.Context) error {
log := createLogger(c)
token, err := getManagementToken(c, log)
if err != nil {
return err
}
var tokenResponse = struct {
Token string `json:"token"`
}{Token: token}
return json.NewEncoder(os.Stdout).Encode(tokenResponse)
}
func buildTailCommand(subcommands []*cli.Command) *cli.Command {
return &cli.Command{
Name: "tail",
Action: Run,
Usage: "Stream logs from a remote cloudflared",
UsageText: "cloudflared tail [tail command options] [TUNNEL-ID]",
Flags: []cli.Flag{
&cli.StringFlag{
Name: "connector-id",
@@ -39,6 +76,23 @@ func Command() *cli.Command {
Value: "",
EnvVars: []string{"TUNNEL_MANAGEMENT_CONNECTOR"},
},
&cli.StringSliceFlag{
Name: "event",
Usage: "Filter by specific Events (cloudflared, http, tcp, udp) otherwise, defaults to send all events",
EnvVars: []string{"TUNNEL_MANAGEMENT_FILTER_EVENTS"},
},
&cli.StringFlag{
Name: "level",
Usage: "Filter by specific log levels (debug, info, warn, error). Filters by debug log level by default.",
EnvVars: []string{"TUNNEL_MANAGEMENT_FILTER_LEVEL"},
Value: "debug",
},
&cli.Float64Flag{
Name: "sample",
Usage: "Sample log events by percentage (0.0 .. 1.0). No sampling by default.",
EnvVars: []string{"TUNNEL_MANAGEMENT_FILTER_SAMPLE"},
Value: 1.0,
},
&cli.StringFlag{
Name: "token",
Usage: "Access token for a specific tunnel",
@@ -61,10 +115,17 @@ func Command() *cli.Command {
&cli.StringFlag{
Name: logger.LogLevelFlag,
Value: "info",
Usage: "Application logging level {debug, info, warn, error, fatal}. ",
Usage: "Application logging level {debug, info, warn, error, fatal}",
EnvVars: []string{"TUNNEL_LOGLEVEL"},
},
&cli.StringFlag{
Name: credentials.OriginCertFlag,
Usage: "Path to the certificate generated for your origin when you run cloudflared login.",
EnvVars: []string{"TUNNEL_ORIGIN_CERT"},
Value: credentials.FindDefaultOriginCertPath(),
},
},
Subcommands: subcommands,
}
}
@@ -113,6 +174,102 @@ func createLogger(c *cli.Context) *zerolog.Logger {
return &log
}
// parseFilters will attempt to parse provided filters to send to with the EventStartStreaming
func parseFilters(c *cli.Context) (*management.StreamingFilters, error) {
var level *management.LogLevel
var events []management.LogEventType
var sample float64
argLevel := c.String("level")
argEvents := c.StringSlice("event")
argSample := c.Float64("sample")
if argLevel != "" {
l, ok := management.ParseLogLevel(argLevel)
if !ok {
return nil, fmt.Errorf("invalid --level filter provided, please use one of the following Log Levels: debug, info, warn, error")
}
level = &l
}
for _, v := range argEvents {
t, ok := management.ParseLogEventType(v)
if !ok {
return nil, fmt.Errorf("invalid --event filter provided, please use one of the following EventTypes: cloudflared, http, tcp, udp")
}
events = append(events, t)
}
if argSample <= 0.0 || argSample > 1.0 {
return nil, fmt.Errorf("invalid --sample value provided, please make sure it is in the range (0.0 .. 1.0)")
}
sample = argSample
if level == nil && len(events) == 0 && argSample != 1.0 {
// When no filters are provided, do not return a StreamingFilters struct
return nil, nil
}
return &management.StreamingFilters{
Level: level,
Events: events,
Sampling: sample,
}, nil
}
// getManagementToken will make a call to the Cloudflare API to acquire a management token for the requested tunnel.
func getManagementToken(c *cli.Context, log *zerolog.Logger) (string, error) {
userCreds, err := credentials.Read(c.String(credentials.OriginCertFlag), log)
if err != nil {
return "", err
}
client, err := userCreds.Client(c.String("api-url"), buildInfo.UserAgent(), log)
if err != nil {
return "", err
}
tunnelIDString := c.Args().First()
if tunnelIDString == "" {
return "", errors.New("no tunnel ID provided")
}
tunnelID, err := uuid.Parse(tunnelIDString)
if err != nil {
return "", errors.New("unable to parse provided tunnel id as a valid UUID")
}
token, err := client.GetManagementToken(tunnelID)
if err != nil {
return "", err
}
return token, nil
}
// buildURL will build the management url to contain the required query parameters to authenticate the request.
func buildURL(c *cli.Context, log *zerolog.Logger) (url.URL, error) {
var err error
managementHostname := c.String("management-hostname")
token := c.String("token")
if token == "" {
token, err = getManagementToken(c, log)
if err != nil {
return url.URL{}, fmt.Errorf("unable to acquire management token for requested tunnel id: %w", err)
}
}
query := url.Values{}
query.Add("access_token", token)
connector := c.String("connector-id")
if connector != "" {
connectorID, err := uuid.Parse(connector)
if err != nil {
return url.URL{}, fmt.Errorf("unabled to parse 'connector-id' flag into a valid UUID: %w", err)
}
query.Add("connector_id", connectorID.String())
}
return url.URL{Scheme: "wss", Host: managementHostname, Path: "/logs", RawQuery: query.Encode()}, nil
}
// Run implements a foreground runner
func Run(c *cli.Context) error {
log := createLogger(c)
@@ -121,12 +278,20 @@ func Run(c *cli.Context) error {
signal.Notify(signals, syscall.SIGTERM, syscall.SIGINT)
defer signal.Stop(signals)
managementHostname := c.String("management-hostname")
token := c.String("token")
u := url.URL{Scheme: "wss", Host: managementHostname, Path: "/logs", RawQuery: "access_token=" + token}
filters, err := parseFilters(c)
if err != nil {
log.Error().Err(err).Msgf("invalid filters provided")
return nil
}
u, err := buildURL(c, log)
if err != nil {
log.Err(err).Msg("unable to construct management request URL")
return nil
}
header := make(http.Header)
header.Add("User-Agent", "cloudflared/"+version)
header.Add("User-Agent", buildInfo.UserAgent())
trace := c.String("trace")
if trace != "" {
header["cf-trace-id"] = []string{trace}
@@ -148,11 +313,17 @@ func Run(c *cli.Context) error {
// Once connection is established, send start_streaming event to begin receiving logs
err = management.WriteEvent(conn, ctx, &management.EventStartStreaming{
ClientEvent: management.ClientEvent{Type: management.StartStreaming},
Filters: filters,
})
if err != nil {
log.Error().Err(err).Msg("unable to request logs from management tunnel")
return nil
}
log.Debug().
Str("tunnel-id", c.Args().First()).
Str("connector-id", c.String("connector-id")).
Interface("filters", filters).
Msg("connected")
readerDone := make(chan struct{})
@@ -187,7 +358,12 @@ func Run(c *cli.Context) error {
}
// Output all the logs received to stdout
for _, l := range logs.Logs {
fmt.Printf("%s %s %s %s\n", l.Timestamp, l.Level, l.Event, l.Message)
fields, err := json.Marshal(l.Fields)
if err != nil {
fields = []byte("unable to parse fields")
log.Debug().Msgf("unable to parse fields from event %+v", l)
}
fmt.Printf("%s %s %s %s %s\n", l.Time, l.Level, l.Event, l.Message, fields)
}
case management.UnknownServerEventType:
fallthrough
+32 -3
View File
@@ -28,6 +28,8 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/updater"
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/credentials"
"github.com/cloudflare/cloudflared/edgediscovery"
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/logger"
@@ -94,6 +96,7 @@ Eg. cloudflared tunnel --url localhost:8080/.
Please note that Quick Tunnels are meant to be ephemeral and should only be used for testing purposes.
For production usage, we recommend creating Named Tunnels. (https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/tunnel-guide/)
`
connectorLabelFlag = "label"
)
var (
@@ -398,7 +401,21 @@ func StartServer(
localRules := []ingress.Rule{}
if features.Contains(features.FeatureManagementLogs) {
mgmt := management.New(c.String("management-hostname"), logger.ManagementLogger.Log, logger.ManagementLogger)
serviceIP := c.String("service-op-ip")
if edgeAddrs, err := edgediscovery.ResolveEdge(log, tunnelConfig.Region, tunnelConfig.EdgeIPVersion); err == nil {
if serviceAddr, err := edgeAddrs.GetAddrForRPC(); err == nil {
serviceIP = serviceAddr.TCP.String()
}
}
mgmt := management.New(
c.String("management-hostname"),
serviceIP,
clientID,
c.String(connectorLabelFlag),
logger.ManagementLogger.Log,
logger.ManagementLogger,
)
localRules = []ingress.Rule{ingress.NewManagementRule(mgmt)}
}
orchestrator, err := orchestration.NewOrchestrator(ctx, orchestratorConfig, tunnelConfig.Tags, localRules, tunnelConfig.Log)
@@ -666,6 +683,11 @@ func tunnelFlags(shouldHide bool) []cli.Flag {
Value: 4,
Hidden: true,
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: connectorLabelFlag,
Usage: "Use this option to give a meaningful label to a specific connector. When a tunnel starts up, a connector id unique to the tunnel is generated. This is a uuid. To make it easier to identify a connector, we will use the hostname of the machine the tunnel is running on along with the connector ID. This option exists if one wants to have more control over what their individual connectors are called.",
Value: "",
}),
altsrc.NewDurationFlag(&cli.DurationFlag{
Name: "grace-period",
Usage: "When cloudflared receives SIGINT/SIGTERM it will stop accepting new requests, wait for in-progress requests to terminate, then shutdown. Waiting for in-progress requests will timeout after this grace period, or when a second SIGTERM/SIGINT is received.",
@@ -751,10 +773,10 @@ func configureCloudflaredFlags(shouldHide bool) []cli.Flag {
Hidden: shouldHide,
},
altsrc.NewStringFlag(&cli.StringFlag{
Name: "origincert",
Name: credentials.OriginCertFlag,
Usage: "Path to the certificate generated for your origin when you run cloudflared login.",
EnvVars: []string{"TUNNEL_ORIGIN_CERT"},
Value: findDefaultOriginCertPath(),
Value: credentials.FindDefaultOriginCertPath(),
Hidden: shouldHide,
}),
altsrc.NewDurationFlag(&cli.DurationFlag{
@@ -906,6 +928,13 @@ func configureProxyFlags(shouldHide bool) []cli.Flag {
Hidden: true,
Value: "management.argotunnel.com",
}),
altsrc.NewStringFlag(&cli.StringFlag{
Name: "service-op-ip",
Usage: "Fallback IP for service operations run by the management service.",
EnvVars: []string{"TUNNEL_SERVICE_OP_IP"},
Hidden: true,
Value: "198.41.200.113:80",
}),
}
return append(flags, sshFlags(shouldHide)...)
}
-72
View File
@@ -3,17 +3,14 @@ package tunnel
import (
"crypto/tls"
"fmt"
"io/ioutil"
mathRand "math/rand"
"net"
"net/netip"
"os"
"path/filepath"
"strings"
"time"
"github.com/google/uuid"
homedir "github.com/mitchellh/go-homedir"
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
@@ -33,7 +30,6 @@ import (
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
const LogFieldOriginCertPath = "originCertPath"
const secretValue = "*****"
var (
@@ -46,18 +42,6 @@ var (
configFlags = []string{"autoupdate-freq", "no-autoupdate", "retries", "protocol", "loglevel", "transport-loglevel", "origincert", "metrics", "metrics-update-freq", "edge-ip-version", "edge-bind-address"}
)
// returns the first path that contains a cert.pem file. If none of the DefaultConfigSearchDirectories
// contains a cert.pem file, return empty string
func findDefaultOriginCertPath() string {
for _, defaultConfigDir := range config.DefaultConfigSearchDirectories() {
originCertPath, _ := homedir.Expand(filepath.Join(defaultConfigDir, config.DefaultCredentialFile))
if ok, _ := config.FileExists(originCertPath); ok {
return originCertPath
}
}
return ""
}
func generateRandomClientID(log *zerolog.Logger) (string, error) {
u, err := uuid.NewRandom()
if err != nil {
@@ -128,62 +112,6 @@ func dnsProxyStandAlone(c *cli.Context, namedTunnel *connection.NamedTunnelPrope
namedTunnel != nil) // named tunnel
}
func findOriginCert(originCertPath string, log *zerolog.Logger) (string, error) {
if originCertPath == "" {
log.Info().Msgf("Cannot determine default origin certificate path. No file %s in %v", config.DefaultCredentialFile, config.DefaultConfigSearchDirectories())
if isRunningFromTerminal() {
log.Error().Msgf("You need to specify the origin certificate path with --origincert option, or set TUNNEL_ORIGIN_CERT environment variable. See %s for more information.", argumentsUrl)
return "", fmt.Errorf("client didn't specify origincert path when running from terminal")
} else {
log.Error().Msgf("You need to specify the origin certificate path by specifying the origincert option in the configuration file, or set TUNNEL_ORIGIN_CERT environment variable. See %s for more information.", serviceUrl)
return "", fmt.Errorf("client didn't specify origincert path")
}
}
var err error
originCertPath, err = homedir.Expand(originCertPath)
if err != nil {
log.Err(err).Msgf("Cannot resolve origin certificate path")
return "", fmt.Errorf("cannot resolve path %s", originCertPath)
}
// Check that the user has acquired a certificate using the login command
ok, err := config.FileExists(originCertPath)
if err != nil {
log.Error().Err(err).Msgf("Cannot check if origin cert exists at path %s", originCertPath)
return "", fmt.Errorf("cannot check if origin cert exists at path %s", originCertPath)
}
if !ok {
log.Error().Msgf(`Cannot find a valid certificate for your origin at the path:
%s
If the path above is wrong, specify the path with the -origincert option.
If you don't have a certificate signed by Cloudflare, run the command:
%s login
`, originCertPath, os.Args[0])
return "", fmt.Errorf("cannot find a valid certificate at the path %s", originCertPath)
}
return originCertPath, nil
}
func readOriginCert(originCertPath string) ([]byte, error) {
// Easier to send the certificate as []byte via RPC than decoding it at this point
originCert, err := ioutil.ReadFile(originCertPath)
if err != nil {
return nil, fmt.Errorf("cannot read %s to load origin certificate", originCertPath)
}
return originCert, nil
}
func getOriginCert(originCertPath string, log *zerolog.Logger) ([]byte, error) {
if originCertPath, err := findOriginCert(originCertPath, log); err != nil {
return nil, err
} else {
return readOriginCert(originCertPath)
}
}
func prepareTunnelConfig(
c *cli.Context,
info *cliutil.BuildInfo,
+4 -3
View File
@@ -5,6 +5,7 @@ import (
"path/filepath"
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/credentials"
"github.com/google/uuid"
"github.com/rs/zerolog"
@@ -56,13 +57,13 @@ func newSearchByID(id uuid.UUID, c *cli.Context, log *zerolog.Logger, fs fileSys
}
func (s searchByID) Path() (string, error) {
originCertPath := s.c.String("origincert")
originCertPath := s.c.String(credentials.OriginCertFlag)
originCertLog := s.log.With().
Str(LogFieldOriginCertPath, originCertPath).
Str("originCertPath", originCertPath).
Logger()
// Fallback to look for tunnel credentials in the origin cert directory
if originCertPath, err := findOriginCert(originCertPath, &originCertLog); err == nil {
if originCertPath, err := credentials.FindOriginCert(originCertPath, &originCertLog); err == nil {
originCertDir := filepath.Dir(originCertPath)
if filePath, err := tunnelFilePath(s.id, originCertDir); err == nil {
if s.fs.validFilePath(filePath) {
+3 -1
View File
@@ -14,6 +14,7 @@ import (
"github.com/cloudflare/cloudflared/cmd/cloudflared/cliutil"
"github.com/cloudflare/cloudflared/config"
"github.com/cloudflare/cloudflared/credentials"
"github.com/cloudflare/cloudflared/logger"
"github.com/cloudflare/cloudflared/token"
)
@@ -52,6 +53,7 @@ func login(c *cli.Context) error {
resourceData, err := token.RunTransfer(
loginURL,
"",
"cert",
"callback",
callbackStoreURL,
@@ -85,7 +87,7 @@ func checkForExistingCert() (string, bool, error) {
if err != nil {
return "", false, err
}
path := filepath.Join(configPath, config.DefaultCredentialFile)
path := filepath.Join(configPath, credentials.DefaultCredentialFile)
fileInfo, err := os.Stat(path)
if err == nil && fileInfo.Size() > 0 {
return path, true, nil
+11 -48
View File
@@ -13,9 +13,9 @@ import (
"github.com/rs/zerolog"
"github.com/urfave/cli/v2"
"github.com/cloudflare/cloudflared/certutil"
"github.com/cloudflare/cloudflared/cfapi"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/credentials"
"github.com/cloudflare/cloudflared/logger"
)
@@ -37,7 +37,7 @@ type subcommandContext struct {
// These fields should be accessed using their respective Getter
tunnelstoreClient cfapi.Client
userCredential *userCredential
userCredential *credentials.User
}
func newSubcommandContext(c *cli.Context) (*subcommandContext, error) {
@@ -56,65 +56,28 @@ func (sc *subcommandContext) credentialFinder(tunnelID uuid.UUID) CredFinder {
return newSearchByID(tunnelID, sc.c, sc.log, sc.fs)
}
type userCredential struct {
cert *certutil.OriginCert
certPath string
}
func (sc *subcommandContext) client() (cfapi.Client, error) {
if sc.tunnelstoreClient != nil {
return sc.tunnelstoreClient, nil
}
credential, err := sc.credential()
cred, err := sc.credential()
if err != nil {
return nil, err
}
userAgent := fmt.Sprintf("cloudflared/%s", buildInfo.Version())
client, err := cfapi.NewRESTClient(
sc.c.String("api-url"),
credential.cert.AccountID,
credential.cert.ZoneID,
credential.cert.APIToken,
userAgent,
sc.log,
)
sc.tunnelstoreClient, err = cred.Client(sc.c.String("api-url"), buildInfo.UserAgent(), sc.log)
if err != nil {
return nil, err
}
sc.tunnelstoreClient = client
return client, nil
return sc.tunnelstoreClient, nil
}
func (sc *subcommandContext) credential() (*userCredential, error) {
func (sc *subcommandContext) credential() (*credentials.User, error) {
if sc.userCredential == nil {
originCertPath := sc.c.String("origincert")
originCertLog := sc.log.With().
Str(LogFieldOriginCertPath, originCertPath).
Logger()
originCertPath, err := findOriginCert(originCertPath, &originCertLog)
uc, err := credentials.Read(sc.c.String(credentials.OriginCertFlag), sc.log)
if err != nil {
return nil, errors.Wrap(err, "Error locating origin cert")
}
blocks, err := readOriginCert(originCertPath)
if err != nil {
return nil, errors.Wrapf(err, "Can't read origin cert from %s", originCertPath)
}
cert, err := certutil.DecodeOriginCert(blocks)
if err != nil {
return nil, errors.Wrap(err, "Error decoding origin cert")
}
if cert.AccountID == "" {
return nil, errors.Errorf(`Origin certificate needs to be refreshed before creating new tunnels.\nDelete %s and run "cloudflared login" to obtain a new cert.`, originCertPath)
}
sc.userCredential = &userCredential{
cert: cert,
certPath: originCertPath,
return nil, err
}
sc.userCredential = uc
}
return sc.userCredential, nil
}
@@ -175,13 +138,13 @@ func (sc *subcommandContext) create(name string, credentialsFilePath string, sec
return nil, err
}
tunnelCredentials := connection.Credentials{
AccountTag: credential.cert.AccountID,
AccountTag: credential.AccountID(),
TunnelSecret: tunnelSecret,
TunnelID: tunnel.ID,
}
usedCertPath := false
if credentialsFilePath == "" {
originCertDir := filepath.Dir(credential.certPath)
originCertDir := filepath.Dir(credential.CertPath())
credentialsFilePath, err = tunnelFilePath(tunnelCredentials.TunnelID, originCertDir)
if err != nil {
return nil, err
@@ -16,6 +16,7 @@ import (
"github.com/cloudflare/cloudflared/cfapi"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/credentials"
)
type mockFileSystem struct {
@@ -37,7 +38,7 @@ func Test_subcommandContext_findCredentials(t *testing.T) {
log *zerolog.Logger
fs fileSystem
tunnelstoreClient cfapi.Client
userCredential *userCredential
userCredential *credentials.User
}
type args struct {
tunnelID uuid.UUID
@@ -249,7 +250,7 @@ func Test_subcommandContext_Delete(t *testing.T) {
isUIEnabled bool
fs fileSystem
tunnelstoreClient *deleteMockTunnelStore
userCredential *userCredential
userCredential *credentials.User
}
type args struct {
tunnelIDs []uuid.UUID
+14
View File
@@ -6,6 +6,7 @@ import (
"os"
"path/filepath"
"runtime"
"strings"
"time"
"github.com/facebookgo/grace/gracenet"
@@ -95,11 +96,24 @@ func CheckForUpdate(options updateOptions) (CheckResult, error) {
url = StagingUpdateURL
}
if runtime.GOOS == "windows" {
cfdPath = encodeWindowsPath(cfdPath)
}
s := NewWorkersService(version, url, cfdPath, Options{IsBeta: options.isBeta,
IsForced: options.isForced, RequestedVersion: options.intendedVersion})
return s.Check()
}
func encodeWindowsPath(path string) string {
// We do this because Windows allows spaces in directories such as
// Program Files but does not allow these directories to be spaced in batch files.
targetPath := strings.Replace(path, "Program Files (x86)", "PROGRA~2", -1)
// This is to do the same in 32 bit systems. We do this second so that the first
// replace is for x86 dirs.
targetPath = strings.Replace(targetPath, "Program Files", "PROGRA~1", -1)
return targetPath
}
func applyUpdate(options updateOptions, update CheckResult) UpdateOutcome {
if update.Version() == "" || options.updateDisabled {
+18 -8
View File
@@ -25,14 +25,13 @@ const (
// rename cloudflared.exe.new to cloudflared.exe
// delete cloudflared.exe.old
// start the service
// delete the batch file
windowsUpdateCommandTemplate = `@echo off
sc stop cloudflared >nul 2>&1
// exit with code 0 if we've reached this point indicating success.
windowsUpdateCommandTemplate = `sc stop cloudflared >nul 2>&1
rename "{{.TargetPath}}" {{.OldName}}
rename "{{.NewPath}}" {{.BinaryName}}
del "{{.OldPath}}"
sc start cloudflared >nul 2>&1
del {{.BatchName}}`
exit /b 0`
batchFileName = "cfd_update.bat"
)
@@ -214,8 +213,9 @@ func isValidChecksum(checksum, filePath string) error {
// writeBatchFile writes a batch file out to disk
// see the dicussion on why it has to be done this way
func writeBatchFile(targetPath string, newPath string, oldPath string) error {
os.Remove(batchFileName) //remove any failed updates before download
f, err := os.Create(batchFileName)
batchFilePath := filepath.Join(filepath.Dir(targetPath), batchFileName)
os.Remove(batchFilePath) //remove any failed updates before download
f, err := os.Create(batchFilePath)
if err != nil {
return err
}
@@ -241,6 +241,16 @@ func writeBatchFile(targetPath string, newPath string, oldPath string) error {
// run each OS command for windows
func runWindowsBatch(batchFile string) error {
cmd := exec.Command("cmd", "/c", batchFile)
return cmd.Start()
defer os.Remove(batchFile)
cmd := exec.Command("cmd", "/C", batchFile)
_, err := cmd.Output()
// Remove the batch file we created. Don't let this interfere with the error
// we report.
if err != nil {
if exitError, ok := err.(*exec.ExitError); ok {
return fmt.Errorf("Error during update : %s;", string(exitError.Stderr))
}
}
return err
}
+32
View File
@@ -2,7 +2,9 @@ import json
import subprocess
from time import sleep
from constants import MANAGEMENT_HOST_NAME
from setup import get_config_from_file
from util import get_tunnel_connector_id
SINGLE_CASE_TIMEOUT = 600
@@ -28,6 +30,36 @@ class CloudflaredCli:
listed = self._run_command(cmd_args, "list")
return json.loads(listed.stdout)
def get_management_token(self, config, config_path):
basecmd = [config.cloudflared_binary]
if config_path is not None:
basecmd += ["--config", str(config_path)]
origincert = get_config_from_file()["origincert"]
if origincert:
basecmd += ["--origincert", origincert]
cmd_args = ["tail", "token", config.get_tunnel_id()]
cmd = basecmd + cmd_args
result = run_subprocess(cmd, "token", self.logger, check=True, capture_output=True, timeout=15)
return json.loads(result.stdout.decode("utf-8").strip())["token"]
def get_management_url(self, path, config, config_path):
access_jwt = self.get_management_token(config, config_path)
connector_id = get_tunnel_connector_id()
return f"https://{MANAGEMENT_HOST_NAME}/{path}?connector_id={connector_id}&access_token={access_jwt}"
def get_management_wsurl(self, path, config, config_path):
access_jwt = self.get_management_token(config, config_path)
connector_id = get_tunnel_connector_id()
return f"wss://{MANAGEMENT_HOST_NAME}/{path}?connector_id={connector_id}&access_token={access_jwt}"
def get_connector_id(self, config):
op = self.get_tunnel_info(config.get_tunnel_id())
connectors = []
for conn in op["conns"]:
connectors.append(conn["id"])
return connectors
def get_tunnel_info(self, tunnel_id):
info = self._run_command(["info", "--output", "json", tunnel_id], "info")
return json.loads(info.stdout)
+8
View File
@@ -0,0 +1,8 @@
cloudflared_binary: "cloudflared"
tunnel: "ae21a96c-24d1-4ce8-a6ba-962cba5976d3"
credentials_file: "/Users/sudarsan/.cloudflared/ae21a96c-24d1-4ce8-a6ba-962cba5976d3.json"
origincert: "/Users/sudarsan/.cloudflared/cert.pem"
ingress:
- hostname: named-tunnel-component-tests.example.com
service: hello_world
- service: http_status:404
+1
View File
@@ -4,6 +4,7 @@ BACKOFF_SECS = 7
MAX_LOG_LINES = 50
PROXY_DNS_PORT = 9053
MANAGEMENT_HOST_NAME = "management.argotunnel.com"
def protocols():
+5 -3
View File
@@ -1,6 +1,8 @@
cloudflare==2.8.15
flaky==3.7.0
pytest==6.2.2
pytest==7.3.1
pytest-asyncio==0.21.0
pyyaml==5.4.1
requests==2.25.1
retrying==1.3.3
requests==2.28.2
retrying==1.3.4
websockets==11.0.1
+3 -3
View File
@@ -74,7 +74,7 @@ def assert_log_to_dir(config, log_dir):
class TestLogging:
def test_logging_to_terminal(self, tmp_path, component_tests_config):
config = component_tests_config()
with start_cloudflared(tmp_path, config, new_process=True) as cloudflared:
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True) as cloudflared:
wait_tunnel_ready(tunnel_url=config.get_url())
assert_log_to_terminal(cloudflared)
@@ -85,7 +85,7 @@ class TestLogging:
"logfile": str(log_file),
}
config = component_tests_config(extra_config)
with start_cloudflared(tmp_path, config, new_process=True, capture_output=False):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False):
wait_tunnel_ready(tunnel_url=config.get_url(), cfd_logs=str(log_file))
assert_log_in_file(log_file)
assert_json_log(log_file)
@@ -98,6 +98,6 @@ class TestLogging:
"log-directory": str(log_dir),
}
config = component_tests_config(extra_config)
with start_cloudflared(tmp_path, config, new_process=True, capture_output=False):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False):
wait_tunnel_ready(tunnel_url=config.get_url(), cfd_logs=str(log_dir))
assert_log_to_dir(config, log_dir)
+2 -2
View File
@@ -12,6 +12,6 @@ class TestPostQuantum:
def test_post_quantum(self, tmp_path, component_tests_config):
config = component_tests_config(self._extra_config())
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--post-quantum"], new_process=True):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["run", "--post-quantum"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(),
require_min_connections=4)
require_min_connections=1)
+1 -1
View File
@@ -25,7 +25,7 @@ def run_test_scenario(tmp_path, component_tests_config, cfd_mode, run_proxy_dns)
if cfd_mode == CfdModes.NAMED:
expect_tunnel = True
pre_args = ["tunnel"]
pre_args = ["tunnel", "--ha-connections", "1"]
args = ["run"]
elif cfd_mode == CfdModes.PROXY_DNS:
expect_proxy_dns = True
+3 -3
View File
@@ -7,7 +7,7 @@ class TestQuickTunnels:
def test_quick_tunnel(self, tmp_path, component_tests_config):
config = component_tests_config(cfd_mode=CfdModes.QUICK, run_proxy_dns=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["--hello-world"], new_process=True):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["--hello-world"], new_process=True):
wait_tunnel_ready(require_min_connections=1)
url = get_quicktunnel_url()
send_requests(url, 3, True)
@@ -15,8 +15,8 @@ class TestQuickTunnels:
def test_quick_tunnel_url(self, tmp_path, component_tests_config):
config = component_tests_config(cfd_mode=CfdModes.QUICK, run_proxy_dns=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["--url", f"http://localhost:{METRICS_PORT}/"], new_process=True):
wait_tunnel_ready()
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["--url", f"http://localhost:{METRICS_PORT}/"], new_process=True):
wait_tunnel_ready(require_min_connections=1)
url = get_quicktunnel_url()
send_requests(url+"/ready", 3, True)
+2 -2
View File
@@ -13,7 +13,7 @@ from util import start_cloudflared, wait_tunnel_ready, check_tunnel_not_connecte
@flaky(max_runs=3, min_passes=1)
class TestReconnect:
default_ha_conns = 4
default_ha_conns = 1
default_reconnect_secs = 15
extra_config = {
"stdin-control": True,
@@ -29,7 +29,7 @@ class TestReconnect:
@pytest.mark.parametrize("protocol", protocols())
def test_named_reconnect(self, tmp_path, component_tests_config, protocol):
config = component_tests_config(self._extra_config(protocol))
with start_cloudflared(tmp_path, config, new_process=True, allow_input=True, capture_output=False) as cloudflared:
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, allow_input=True, capture_output=False) as cloudflared:
# Repeat the test multiple times because some issues only occur after multiple reconnects
self.assert_reconnect(config, cloudflared, 5)
+182
View File
@@ -0,0 +1,182 @@
#!/usr/bin/env python
import asyncio
import json
import pytest
import requests
import websockets
from websockets.client import connect, WebSocketClientProtocol
from conftest import CfdModes
from constants import MAX_RETRIES, BACKOFF_SECS
from retrying import retry
from cli import CloudflaredCli
from util import LOGGER, start_cloudflared, write_config, wait_tunnel_ready
class TestTail:
@pytest.mark.asyncio
async def test_start_stop_streaming(self, tmp_path, component_tests_config):
"""
Validates that a websocket connection to management.argotunnel.com/logs can be opened
with the access token and start and stop streaming on-demand.
"""
print("test_start_stop_streaming")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
async with connect(url, open_timeout=5, close_timeout=3) as websocket:
await websocket.send('{"type": "start_streaming"}')
await websocket.send('{"type": "stop_streaming"}')
await websocket.send('{"type": "start_streaming"}')
await websocket.send('{"type": "stop_streaming"}')
@pytest.mark.asyncio
async def test_streaming_logs(self, tmp_path, component_tests_config):
"""
Validates that a streaming logs connection will stream logs
"""
print("test_streaming_logs")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
# send start_streaming
await websocket.send('{"type": "start_streaming"}')
# send some http requests to the tunnel to trigger some logs
await generate_and_validate_http_events(websocket, config.get_url(), 10)
# send stop_streaming
await websocket.send('{"type": "stop_streaming"}')
@pytest.mark.asyncio
async def test_streaming_logs_filters(self, tmp_path, component_tests_config):
"""
Validates that a streaming logs connection will stream logs
but not http when filters applied.
"""
print("test_streaming_logs_filters")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
# send start_streaming with tcp logs only
await websocket.send(json.dumps({
"type": "start_streaming",
"filters": {
"events": ["tcp"],
"level": "debug"
}
}))
# don't expect any http logs
await generate_and_validate_no_log_event(websocket, config.get_url())
# send stop_streaming
await websocket.send('{"type": "stop_streaming"}')
@pytest.mark.asyncio
async def test_streaming_logs_sampling(self, tmp_path, component_tests_config):
"""
Validates that a streaming logs connection will stream logs with sampling.
"""
print("test_streaming_logs_sampling")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
# send start_streaming with info logs only
await websocket.send(json.dumps({
"type": "start_streaming",
"filters": {
"sampling": 0.5
}
}))
# don't expect any http logs
count = await generate_and_validate_http_events(websocket, config.get_url(), 10)
assert count < (10 * 2) # There are typically always two log lines for http requests (request and response)
# send stop_streaming
await websocket.send('{"type": "stop_streaming"}')
@pytest.mark.asyncio
async def test_streaming_logs_actor_override(self, tmp_path, component_tests_config):
"""
Validates that a streaming logs session can be overriden by the same actor
"""
print("test_streaming_logs_actor_override")
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(), require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
url = cfd_cli.get_management_wsurl("logs", config, config_path)
task = asyncio.ensure_future(start_streaming_to_be_remotely_closed(url))
override_task = asyncio.ensure_future(start_streaming_override(url))
await asyncio.wait([task, override_task])
assert task.exception() == None, task.exception()
assert override_task.exception() == None, override_task.exception()
async def start_streaming_to_be_remotely_closed(url):
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
try:
await websocket.send(json.dumps({"type": "start_streaming"}))
await asyncio.sleep(10)
assert websocket.closed, "expected this request to be forcibly closed by the override"
except websockets.ConnectionClosed:
# we expect the request to be closed
pass
async def start_streaming_override(url):
# wait for the first connection to be established
await asyncio.sleep(1)
async with connect(url, open_timeout=5, close_timeout=5) as websocket:
await websocket.send(json.dumps({"type": "start_streaming"}))
await asyncio.sleep(1)
await websocket.send(json.dumps({"type": "stop_streaming"}))
await asyncio.sleep(1)
# Every http request has two log lines sent
async def generate_and_validate_http_events(websocket: WebSocketClientProtocol, url: str, count_send: int):
for i in range(count_send):
send_request(url)
# There are typically always two log lines for http requests (request and response)
count = 0
while True:
try:
req_line = await asyncio.wait_for(websocket.recv(), 2)
log_line = json.loads(req_line)
assert log_line["type"] == "logs"
assert log_line["logs"][0]["event"] == "http"
count += 1
except asyncio.TimeoutError:
# ignore timeout from waiting for recv
break
return count
# Every http request has two log lines sent
async def generate_and_validate_no_log_event(websocket: WebSocketClientProtocol, url: str):
send_request(url)
try:
# wait for 5 seconds and make sure we hit the timeout and not recv any events
req_line = await asyncio.wait_for(websocket.recv(), 5)
assert req_line == None, "expected no logs for the specified filters"
except asyncio.TimeoutError:
pass
@retry(stop_max_attempt_number=MAX_RETRIES, wait_fixed=BACKOFF_SECS * 1000)
def send_request(url, headers={}):
with requests.Session() as s:
resp = s.get(url, timeout=BACKOFF_SECS, headers=headers)
assert resp.status_code == 200, f"{url} returned {resp}"
return resp.status_code == 200
+3 -3
View File
@@ -34,7 +34,7 @@ class TestTermination:
def test_graceful_shutdown(self, tmp_path, component_tests_config, signal, protocol):
config = component_tests_config(self._extra_config(protocol))
with start_cloudflared(
tmp_path, config, new_process=True, capture_output=False) as cloudflared:
tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False) as cloudflared:
wait_tunnel_ready(tunnel_url=config.get_url())
connected = threading.Condition()
@@ -56,7 +56,7 @@ class TestTermination:
def test_shutdown_once_no_connection(self, tmp_path, component_tests_config, signal, protocol):
config = component_tests_config(self._extra_config(protocol))
with start_cloudflared(
tmp_path, config, new_process=True, capture_output=False) as cloudflared:
tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False) as cloudflared:
wait_tunnel_ready(tunnel_url=config.get_url())
connected = threading.Condition()
@@ -76,7 +76,7 @@ class TestTermination:
def test_no_connection_shutdown(self, tmp_path, component_tests_config, signal, protocol):
config = component_tests_config(self._extra_config(protocol))
with start_cloudflared(
tmp_path, config, new_process=True, capture_output=False) as cloudflared:
tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], new_process=True, capture_output=False) as cloudflared:
wait_tunnel_ready(tunnel_url=config.get_url())
with self.within_grace_period():
self.terminate_by_signal(cloudflared, signal)
+43 -9
View File
@@ -3,7 +3,9 @@ import requests
from conftest import CfdModes
from constants import METRICS_PORT, MAX_RETRIES, BACKOFF_SECS
from retrying import retry
from util import LOGGER, start_cloudflared, wait_tunnel_ready, send_requests
from cli import CloudflaredCli
from util import LOGGER, write_config, start_cloudflared, wait_tunnel_ready, send_requests
import platform
class TestTunnel:
'''Test tunnels with no ingress rules from config.yaml but ingress rules from CLI only'''
@@ -11,15 +13,47 @@ class TestTunnel:
def test_tunnel_hello_world(self, tmp_path, component_tests_config):
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--hello-world"], new_process=True):
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(),
require_min_connections=4)
require_min_connections=1)
"""
test_get_host_details does the following:
1. It gets a management token from Tunnelstore using cloudflared tail token <tunnel_id>
2. It gets the connector_id after starting a cloudflare tunnel
3. It sends a request to the management host with the connector_id and management token
4. Asserts that the response has a hostname and ip.
"""
def test_get_host_details(self, tmp_path, component_tests_config):
# TUN-7377 : wait_tunnel_ready does not work properly in windows.
# Skipping this test for windows for now and will address it as part of tun-7377
if platform.system() == "Windows":
return
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
headers = {}
headers["Content-Type"] = "application/json"
config_path = write_config(tmp_path, config.full_config)
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1", "--label" , "test"], cfd_args=["run", "--hello-world"], new_process=True):
wait_tunnel_ready(tunnel_url=config.get_url(),
require_min_connections=1)
cfd_cli = CloudflaredCli(config, config_path, LOGGER)
connector_id = cfd_cli.get_connector_id(config)[0]
url = cfd_cli.get_management_url("host_details", config, config_path)
resp = send_request(url, headers=headers)
# Assert response json.
assert resp.status_code == 200, "Expected cloudflared to return 200 for host details"
assert resp.json()["hostname"] == "custom:test", "Expected cloudflared to return hostname"
assert resp.json()["ip"] != "", "Expected cloudflared to return ip"
assert resp.json()["connector_id"] == connector_id, "Expected cloudflared to return connector_id"
def test_tunnel_url(self, tmp_path, component_tests_config):
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["run", "--url", f"http://localhost:{METRICS_PORT}/"], new_process=True):
wait_tunnel_ready(require_min_connections=4)
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["run", "--url", f"http://localhost:{METRICS_PORT}/"], new_process=True):
wait_tunnel_ready(require_min_connections=1)
send_requests(config.get_url()+"/ready", 3, True)
def test_tunnel_no_ingress(self, tmp_path, component_tests_config):
@@ -29,8 +63,8 @@ class TestTunnel:
'''
config = component_tests_config(cfd_mode=CfdModes.NAMED, run_proxy_dns=False, provide_ingress=False)
LOGGER.debug(config)
with start_cloudflared(tmp_path, config, cfd_args=["run"], new_process=True):
wait_tunnel_ready(require_min_connections=4)
with start_cloudflared(tmp_path, config, cfd_pre_args=["tunnel", "--ha-connections", "1"], cfd_args=["run"], new_process=True):
wait_tunnel_ready(require_min_connections=1)
resp = send_request(config.get_url()+"/")
assert resp.status_code == 503, "Expected cloudflared to return 503 for all requests with no ingress defined"
resp = send_request(config.get_url()+"/test")
@@ -38,6 +72,6 @@ class TestTunnel:
@retry(stop_max_attempt_number=MAX_RETRIES, wait_fixed=BACKOFF_SECS * 1000)
def send_request(url):
def send_request(url, headers={}):
with requests.Session() as s:
return s.get(url, timeout=BACKOFF_SECS)
return s.get(url, timeout=BACKOFF_SECS, headers=headers)
+4 -3
View File
@@ -9,6 +9,7 @@ import pytest
import requests
import yaml
import json
from retrying import retry
from constants import METRICS_PORT, MAX_RETRIES, BACKOFF_SECS
@@ -48,7 +49,6 @@ def start_cloudflared(directory, config, cfd_args=["run"], cfd_pre_args=["tunnel
# By setting check=True, it will raise an exception if the process exits with non-zero exit code
return subprocess.run(cmd, check=expect_success, capture_output=capture_output)
def cloudflared_cmd(config, config_path, cfd_args, cfd_pre_args, root):
cmd = []
if root:
@@ -106,13 +106,14 @@ def inner_wait_tunnel_ready(tunnel_url=None, require_min_connections=1):
with requests.Session() as s:
resp = send_request(s, metrics_url, True)
assert resp.json()["readyConnections"] >= require_min_connections, \
ready_connections = resp.json()["readyConnections"]
assert ready_connections >= require_min_connections, \
f"Ready endpoint returned {resp.json()} but we expect at least {require_min_connections} connections"
if tunnel_url is not None:
send_request(s, tunnel_url, True)
def _log_cloudflared_logs(cfd_logs):
log_file = cfd_logs
if os.path.isdir(cfd_logs):
-2
View File
@@ -39,8 +39,6 @@ var (
)
const (
DefaultCredentialFile = "cert.pem"
// BastionFlag is to enable bastion, or jump host, operation
BastionFlag = "bastion"
)
+7 -3
View File
@@ -2,13 +2,13 @@ package connection
import (
"context"
"fmt"
"io"
"net"
"time"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/management"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
)
@@ -85,7 +85,7 @@ func (c *controlStream) ServeControlStream(
return err
}
c.observer.logServerInfo(c.connIndex, registrationDetails.Location, c.edgeAddress, fmt.Sprintf("Connection %s registered with protocol: %s", registrationDetails.UUID, c.protocol))
c.observer.logConnected(registrationDetails.UUID, c.connIndex, registrationDetails.Location, c.edgeAddress, c.protocol)
c.observer.sendConnectedEvent(c.connIndex, c.protocol, registrationDetails.Location)
c.connectedFuse.Connected()
@@ -116,7 +116,11 @@ func (c *controlStream) waitForUnregister(ctx context.Context, rpcClient NamedTu
c.observer.sendUnregisteringEvent(c.connIndex)
rpcClient.GracefulShutdown(ctx, c.gracePeriod)
c.observer.log.Info().Uint8(LogFieldConnIndex, c.connIndex).Msg("Unregistered tunnel connection")
c.observer.log.Info().
Int(management.EventTypeKey, int(management.Cloudflared)).
Uint8(LogFieldConnIndex, c.connIndex).
IPAddr(LogFieldIPAddress, c.edgeAddress).
Msg("Unregistered tunnel connection")
}
func (c *controlStream) IsStopped() bool {
+10 -2
View File
@@ -4,12 +4,17 @@ import (
"net"
"strings"
"github.com/google/uuid"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/management"
)
const (
LogFieldConnectionID = "connection"
LogFieldLocation = "location"
LogFieldIPAddress = "ip"
LogFieldProtocol = "protocol"
observerChannelBufferSize = 16
)
@@ -41,13 +46,16 @@ func (o *Observer) RegisterSink(sink EventSink) {
o.addSinkChan <- sink
}
func (o *Observer) logServerInfo(connIndex uint8, location string, address net.IP, msg string) {
func (o *Observer) logConnected(connectionID uuid.UUID, connIndex uint8, location string, address net.IP, protocol Protocol) {
o.sendEvent(Event{Index: connIndex, EventType: Connected, Location: location})
o.log.Info().
Int(management.EventTypeKey, int(management.Cloudflared)).
Str(LogFieldConnectionID, connectionID.String()).
Uint8(LogFieldConnIndex, connIndex).
Str(LogFieldLocation, location).
IPAddr(LogFieldIPAddress, address).
Msg(msg)
Str(LogFieldProtocol, protocol.String()).
Msg("Registered tunnel connection")
o.metrics.registerServerLocation(uint8ToString(connIndex), location)
}
+16 -5
View File
@@ -25,6 +25,7 @@ import (
"github.com/cloudflare/cloudflared/datagramsession"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/packet"
quicpogs "github.com/cloudflare/cloudflared/quic"
"github.com/cloudflare/cloudflared/tracing"
@@ -298,11 +299,12 @@ func (q *QUICConnection) RegisterUdpSession(ctx context.Context, sessionID uuid.
attribute.String("session-id", sessionID.String()),
attribute.String("dst", fmt.Sprintf("%s:%d", dstIP, dstPort)),
))
log := q.logger.With().Int(management.EventTypeKey, int(management.UDP)).Logger()
// Each session is a series of datagram from an eyeball to a dstIP:dstPort.
// (src port, dst IP, dst port) uniquely identifies a session, so it needs a dedicated connected socket.
originProxy, err := ingress.DialUDP(dstIP, dstPort)
if err != nil {
q.logger.Err(err).Msgf("Failed to create udp proxy to %s:%d", dstIP, dstPort)
log.Err(err).Msgf("Failed to create udp proxy to %s:%d", dstIP, dstPort)
tracing.EndWithErrorStatus(registerSpan, err)
return nil, err
}
@@ -313,14 +315,18 @@ func (q *QUICConnection) RegisterUdpSession(ctx context.Context, sessionID uuid.
session, err := q.sessionManager.RegisterSession(ctx, sessionID, originProxy)
if err != nil {
q.logger.Err(err).Str("sessionID", sessionID.String()).Msgf("Failed to register udp session")
log.Err(err).Str("sessionID", sessionID.String()).Msgf("Failed to register udp session")
tracing.EndWithErrorStatus(registerSpan, err)
return nil, err
}
go q.serveUDPSession(session, closeAfterIdleHint)
q.logger.Debug().Str("sessionID", sessionID.String()).Str("src", originProxy.LocalAddr().String()).Str("dst", fmt.Sprintf("%s:%d", dstIP, dstPort)).Msgf("Registered session")
log.Debug().
Str("sessionID", sessionID.String()).
Str("src", originProxy.LocalAddr().String()).
Str("dst", fmt.Sprintf("%s:%d", dstIP, dstPort)).
Msgf("Registered session")
tracing.End(registerSpan)
resp := tunnelpogs.RegisterUdpSessionResponse{
@@ -341,7 +347,10 @@ func (q *QUICConnection) serveUDPSession(session *datagramsession.Session, close
q.closeUDPSession(ctx, session.ID, "terminated without error")
}
}
q.logger.Debug().Err(err).Str("sessionID", session.ID.String()).Msg("Session terminated")
q.logger.Debug().Err(err).
Int(management.EventTypeKey, int(management.UDP)).
Str("sessionID", session.ID.String()).
Msg("Session terminated")
}
// closeUDPSession first unregisters the session from session manager, then it tries to unregister from edge
@@ -351,7 +360,9 @@ func (q *QUICConnection) closeUDPSession(ctx context.Context, sessionID uuid.UUI
if err != nil {
// Log this at debug because this is not an error if session was closed due to lost connection
// with edge
q.logger.Debug().Err(err).Str("sessionID", sessionID.String()).
q.logger.Debug().Err(err).
Int(management.EventTypeKey, int(management.UDP)).
Str("sessionID", sessionID.String()).
Msgf("Failed to open quic stream to unregister udp session with edge")
return
}
+83
View File
@@ -0,0 +1,83 @@
package credentials
import (
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/cfapi"
)
const (
logFieldOriginCertPath = "originCertPath"
)
type User struct {
cert *OriginCert
certPath string
}
func (c User) AccountID() string {
return c.cert.AccountID
}
func (c User) ZoneID() string {
return c.cert.ZoneID
}
func (c User) APIToken() string {
return c.cert.APIToken
}
func (c User) CertPath() string {
return c.certPath
}
// Client uses the user credentials to create a Cloudflare API client
func (c *User) Client(apiURL string, userAgent string, log *zerolog.Logger) (cfapi.Client, error) {
if apiURL == "" {
return nil, errors.New("An api-url was not provided for the Cloudflare API client")
}
client, err := cfapi.NewRESTClient(
apiURL,
c.cert.AccountID,
c.cert.ZoneID,
c.cert.APIToken,
userAgent,
log,
)
if err != nil {
return nil, err
}
return client, nil
}
// Read will load and read the origin cert.pem to load the user credentials
func Read(originCertPath string, log *zerolog.Logger) (*User, error) {
originCertLog := log.With().
Str(logFieldOriginCertPath, originCertPath).
Logger()
originCertPath, err := FindOriginCert(originCertPath, &originCertLog)
if err != nil {
return nil, errors.Wrap(err, "Error locating origin cert")
}
blocks, err := readOriginCert(originCertPath)
if err != nil {
return nil, errors.Wrapf(err, "Can't read origin cert from %s", originCertPath)
}
cert, err := decodeOriginCert(blocks)
if err != nil {
return nil, errors.Wrap(err, "Error decoding origin cert")
}
if cert.AccountID == "" {
return nil, errors.Errorf(`Origin certificate needs to be refreshed before creating new tunnels.\nDelete %s and run "cloudflared login" to obtain a new cert.`, originCertPath)
}
return &User{
cert: cert,
certPath: originCertPath,
}, nil
}
+38
View File
@@ -0,0 +1,38 @@
package credentials
import (
"io/fs"
"os"
"path"
"testing"
"github.com/stretchr/testify/require"
)
func TestCredentialsRead(t *testing.T) {
file, err := os.ReadFile("test-cloudflare-tunnel-cert-json.pem")
require.NoError(t, err)
dir := t.TempDir()
certPath := path.Join(dir, originCertFile)
os.WriteFile(certPath, file, fs.ModePerm)
user, err := Read(certPath, &nopLog)
require.NoError(t, err)
require.Equal(t, certPath, user.CertPath())
require.Equal(t, "test-service-key", user.APIToken())
require.Equal(t, "7b0a4d77dfb881c1a3b7d61ea9443e19", user.ZoneID())
require.Equal(t, "abcdabcdabcdabcd1234567890abcdef", user.AccountID())
}
func TestCredentialsClient(t *testing.T) {
user := User{
certPath: "/tmp/cert.pem",
cert: &OriginCert{
ZoneID: "7b0a4d77dfb881c1a3b7d61ea9443e19",
AccountID: "abcdabcdabcdabcd1234567890abcdef",
APIToken: "test-service-key",
},
}
client, err := user.Client("example.com", "cloudflared/test", &nopLog)
require.NoError(t, err)
require.NotNil(t, client)
}
+130
View File
@@ -0,0 +1,130 @@
package credentials
import (
"encoding/json"
"encoding/pem"
"fmt"
"os"
"path/filepath"
"github.com/mitchellh/go-homedir"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/config"
)
const (
DefaultCredentialFile = "cert.pem"
OriginCertFlag = "origincert"
)
type namedTunnelToken struct {
ZoneID string `json:"zoneID"`
AccountID string `json:"accountID"`
APIToken string `json:"apiToken"`
}
type OriginCert struct {
ZoneID string
APIToken string
AccountID string
}
// FindDefaultOriginCertPath returns the first path that contains a cert.pem file. If none of the
// DefaultConfigSearchDirectories contains a cert.pem file, return empty string
func FindDefaultOriginCertPath() string {
for _, defaultConfigDir := range config.DefaultConfigSearchDirectories() {
originCertPath, _ := homedir.Expand(filepath.Join(defaultConfigDir, DefaultCredentialFile))
if ok := fileExists(originCertPath); ok {
return originCertPath
}
}
return ""
}
func decodeOriginCert(blocks []byte) (*OriginCert, error) {
if len(blocks) == 0 {
return nil, fmt.Errorf("Cannot decode empty certificate")
}
originCert := OriginCert{}
block, rest := pem.Decode(blocks)
for {
if block == nil {
break
}
switch block.Type {
case "PRIVATE KEY", "CERTIFICATE":
// this is for legacy purposes.
break
case "ARGO TUNNEL TOKEN":
if originCert.ZoneID != "" || originCert.APIToken != "" {
return nil, fmt.Errorf("Found multiple tokens in the certificate")
}
// The token is a string,
// Try the newer JSON format
ntt := namedTunnelToken{}
if err := json.Unmarshal(block.Bytes, &ntt); err == nil {
originCert.ZoneID = ntt.ZoneID
originCert.APIToken = ntt.APIToken
originCert.AccountID = ntt.AccountID
}
default:
return nil, fmt.Errorf("Unknown block %s in the certificate", block.Type)
}
block, rest = pem.Decode(rest)
}
if originCert.ZoneID == "" || originCert.APIToken == "" {
return nil, fmt.Errorf("Missing token in the certificate")
}
return &originCert, nil
}
func readOriginCert(originCertPath string) ([]byte, error) {
originCert, err := os.ReadFile(originCertPath)
if err != nil {
return nil, fmt.Errorf("cannot read %s to load origin certificate", originCertPath)
}
return originCert, nil
}
// FindOriginCert will check to make sure that the certificate exists at the specified file path.
func FindOriginCert(originCertPath string, log *zerolog.Logger) (string, error) {
if originCertPath == "" {
log.Error().Msgf("Cannot determine default origin certificate path. No file %s in %v. You need to specify the origin certificate path by specifying the origincert option in the configuration file, or set TUNNEL_ORIGIN_CERT environment variable", DefaultCredentialFile, config.DefaultConfigSearchDirectories())
return "", fmt.Errorf("client didn't specify origincert path")
}
var err error
originCertPath, err = homedir.Expand(originCertPath)
if err != nil {
log.Err(err).Msgf("Cannot resolve origin certificate path")
return "", fmt.Errorf("cannot resolve path %s", originCertPath)
}
// Check that the user has acquired a certificate using the login command
ok := fileExists(originCertPath)
if !ok {
log.Error().Msgf(`Cannot find a valid certificate for your origin at the path:
%s
If the path above is wrong, specify the path with the -origincert option.
If you don't have a certificate signed by Cloudflare, run the command:
cloudflared login
`, originCertPath)
return "", fmt.Errorf("cannot find a valid certificate at the path %s", originCertPath)
}
return originCertPath, nil
}
// FileExists checks to see if a file exist at the provided path.
func fileExists(path string) bool {
fileStat, err := os.Stat(path)
if err != nil {
return false
}
return !fileStat.IsDir()
}
+110
View File
@@ -0,0 +1,110 @@
package credentials
import (
"fmt"
"io/fs"
"os"
"path"
"testing"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
const (
originCertFile = "cert.pem"
)
var (
nopLog = zerolog.Nop().With().Logger()
)
func TestLoadOriginCert(t *testing.T) {
cert, err := decodeOriginCert([]byte{})
assert.Equal(t, fmt.Errorf("Cannot decode empty certificate"), err)
assert.Nil(t, cert)
blocks, err := os.ReadFile("test-cert-unknown-block.pem")
assert.NoError(t, err)
cert, err = decodeOriginCert(blocks)
assert.Equal(t, fmt.Errorf("Unknown block RSA PRIVATE KEY in the certificate"), err)
assert.Nil(t, cert)
}
func TestJSONArgoTunnelTokenEmpty(t *testing.T) {
blocks, err := os.ReadFile("test-cert-no-token.pem")
assert.NoError(t, err)
cert, err := decodeOriginCert(blocks)
assert.Equal(t, fmt.Errorf("Missing token in the certificate"), err)
assert.Nil(t, cert)
}
func TestJSONArgoTunnelToken(t *testing.T) {
// The given cert's Argo Tunnel Token was generated by base64 encoding this JSON:
// {
// "zoneID": "7b0a4d77dfb881c1a3b7d61ea9443e19",
// "apiToken": "test-service-key",
// "accountID": "abcdabcdabcdabcd1234567890abcdef"
// }
CloudflareTunnelTokenTest(t, "test-cloudflare-tunnel-cert-json.pem")
}
func CloudflareTunnelTokenTest(t *testing.T, path string) {
blocks, err := os.ReadFile(path)
assert.NoError(t, err)
cert, err := decodeOriginCert(blocks)
assert.NoError(t, err)
assert.NotNil(t, cert)
assert.Equal(t, "7b0a4d77dfb881c1a3b7d61ea9443e19", cert.ZoneID)
key := "test-service-key"
assert.Equal(t, key, cert.APIToken)
}
type mockFile struct {
path string
data []byte
err error
}
type mockFileSystem struct {
files map[string]mockFile
}
func newMockFileSystem(files ...mockFile) *mockFileSystem {
fs := mockFileSystem{map[string]mockFile{}}
for _, f := range files {
fs.files[f.path] = f
}
return &fs
}
func (fs *mockFileSystem) ReadFile(path string) ([]byte, error) {
if f, ok := fs.files[path]; ok {
return f.data, f.err
}
return nil, os.ErrNotExist
}
func (fs *mockFileSystem) ValidFilePath(path string) bool {
_, exists := fs.files[path]
return exists
}
func TestFindOriginCert_Valid(t *testing.T) {
file, err := os.ReadFile("test-cloudflare-tunnel-cert-json.pem")
require.NoError(t, err)
dir := t.TempDir()
certPath := path.Join(dir, originCertFile)
os.WriteFile(certPath, file, fs.ModePerm)
path, err := FindOriginCert(certPath, &nopLog)
require.NoError(t, err)
require.Equal(t, certPath, path)
}
func TestFindOriginCert_Missing(t *testing.T) {
dir := t.TempDir()
certPath := path.Join(dir, originCertFile)
_, err := FindOriginCert(certPath, &nopLog)
require.Error(t, err)
}
+4 -1
View File
@@ -9,6 +9,7 @@ import (
"github.com/google/uuid"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/packet"
)
@@ -120,7 +121,9 @@ func (m *manager) registerSession(ctx context.Context, registration *registerSes
}
func (m *manager) newSession(id uuid.UUID, dstConn io.ReadWriteCloser) *Session {
logger := m.log.With().Str("sessionID", id.String()).Logger()
logger := m.log.With().
Int(management.EventTypeKey, int(management.UDP)).
Str("sessionID", id.String()).Logger()
return &Session{
ID: id,
sendFunc: m.sendFunc,
+19 -7
View File
@@ -9,6 +9,8 @@ import (
"github.com/pkg/errors"
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/management"
)
const (
@@ -108,16 +110,20 @@ var friendlyDNSErrorLines = []string{
// EdgeDiscovery implements HA service discovery lookup.
func edgeDiscovery(log *zerolog.Logger, srvService string) ([][]*EdgeAddr, error) {
log.Debug().Str("domain", "_"+srvService+"._"+srvProto+"."+srvName).Msg("looking up edge SRV record")
logger := log.With().Int(management.EventTypeKey, int(management.Cloudflared)).Logger()
logger.Debug().
Int(management.EventTypeKey, int(management.Cloudflared)).
Str("domain", "_"+srvService+"._"+srvProto+"."+srvName).
Msg("edge discovery: looking up edge SRV record")
_, addrs, err := netLookupSRV(srvService, srvProto, srvName)
if err != nil {
_, fallbackAddrs, fallbackErr := fallbackLookupSRV(srvService, srvProto, srvName)
if fallbackErr != nil || len(fallbackAddrs) == 0 {
// use the original DNS error `err` in messages, not `fallbackErr`
log.Err(err).Msg("Error looking up Cloudflare edge IPs: the DNS query failed")
logger.Err(err).Msg("edge discovery: error looking up Cloudflare edge IPs: the DNS query failed")
for _, s := range friendlyDNSErrorLines {
log.Error().Msg(s)
logger.Error().Msg(s)
}
return nil, errors.Wrapf(err, "Could not lookup srv records on _%v._%v.%v", srvService, srvProto, srvName)
}
@@ -131,9 +137,13 @@ func edgeDiscovery(log *zerolog.Logger, srvService string) ([][]*EdgeAddr, error
if err != nil {
return nil, err
}
for _, e := range edgeAddrs {
log.Debug().Msgf("Edge Address: %+v", *e)
logAddrs := make([]string, len(edgeAddrs))
for i, e := range edgeAddrs {
logAddrs[i] = e.UDP.IP.String()
}
logger.Debug().
Strs("addresses", logAddrs).
Msg("edge discovery: resolved edge addresses")
resolvedAddrPerCNAME = append(resolvedAddrPerCNAME, edgeAddrs)
}
@@ -188,13 +198,15 @@ func ResolveAddrs(addrs []string, log *zerolog.Logger) (resolved []*EdgeAddr) {
for _, addr := range addrs {
tcpAddr, err := net.ResolveTCPAddr("tcp", addr)
if err != nil {
log.Error().Str(logFieldAddress, addr).Err(err).Msg("failed to resolve to TCP address")
log.Error().Int(management.EventTypeKey, int(management.Cloudflared)).
Str(logFieldAddress, addr).Err(err).Msg("edge discovery: failed to resolve to TCP address")
continue
}
udpAddr, err := net.ResolveUDPAddr("udp", addr)
if err != nil {
log.Error().Str(logFieldAddress, addr).Err(err).Msg("failed to resolve to UDP address")
log.Error().Int(management.EventTypeKey, int(management.Cloudflared)).
Str(logFieldAddress, addr).Err(err).Msg("edge discovery: failed to resolve to UDP address")
continue
}
version := V6
+21 -17
View File
@@ -6,6 +6,7 @@ import (
"github.com/rs/zerolog"
"github.com/cloudflare/cloudflared/edgediscovery/allregions"
"github.com/cloudflare/cloudflared/management"
)
const (
@@ -74,33 +75,35 @@ func (ed *Edge) GetAddrForRPC() (*allregions.EdgeAddr, error) {
// GetAddr gives this proxy connection an edge Addr. Prefer Addrs this connection has already used.
func (ed *Edge) GetAddr(connIndex int) (*allregions.EdgeAddr, error) {
log := ed.log.With().Int(LogFieldConnIndex, connIndex).Logger()
log := ed.log.With().
Int(LogFieldConnIndex, connIndex).
Int(management.EventTypeKey, int(management.Cloudflared)).
Logger()
ed.Lock()
defer ed.Unlock()
// If this connection has already used an edge addr, return it.
if addr := ed.regions.AddrUsedBy(connIndex); addr != nil {
log.Debug().Msg("edgediscovery - GetAddr: Returning same address back to proxy connection")
log.Debug().IPAddr(LogFieldIPAddress, addr.UDP.IP).Msg("edge discovery: returning same edge address back to pool")
return addr, nil
}
// Otherwise, give it an unused one
addr := ed.regions.GetUnusedAddr(nil, connIndex)
if addr == nil {
log.Debug().Msg("edgediscovery - GetAddr: No addresses left to give proxy connection")
log.Debug().Msg("edge discovery: no addresses left in pool to give proxy connection")
return nil, errNoAddressesLeft
}
log = ed.log.With().
Int(LogFieldConnIndex, connIndex).
IPAddr(LogFieldIPAddress, addr.UDP.IP).Logger()
log.Debug().Msgf("edgediscovery - GetAddr: Giving connection its new address")
log.Debug().IPAddr(LogFieldIPAddress, addr.UDP.IP).Msg("edge discovery: giving new address to connection")
return addr, nil
}
// GetDifferentAddr gives back the proxy connection's edge Addr and uses a new one.
func (ed *Edge) GetDifferentAddr(connIndex int, hasConnectivityError bool) (*allregions.EdgeAddr, error) {
log := ed.log.With().Int(LogFieldConnIndex, connIndex).Logger()
log := ed.log.With().
Int(LogFieldConnIndex, connIndex).
Int(management.EventTypeKey, int(management.Cloudflared)).
Logger()
ed.Lock()
defer ed.Unlock()
@@ -110,14 +113,14 @@ func (ed *Edge) GetDifferentAddr(connIndex int, hasConnectivityError bool) (*all
}
addr := ed.regions.GetUnusedAddr(oldAddr, connIndex)
if addr == nil {
log.Debug().Msg("edgediscovery - GetDifferentAddr: No addresses left to give proxy connection")
log.Debug().Msg("edge discovery: no addresses left in pool to give proxy connection")
// note: if oldAddr were not nil, it will become available on the next iteration
return nil, errNoAddressesLeft
}
log = ed.log.With().
Int(LogFieldConnIndex, connIndex).
IPAddr(LogFieldIPAddress, addr.UDP.IP).Logger()
log.Debug().Msgf("edgediscovery - GetDifferentAddr: Giving connection its new address from the address list: %v", ed.regions.AvailableAddrs())
log.Debug().
IPAddr(LogFieldIPAddress, addr.UDP.IP).
Int("available", ed.regions.AvailableAddrs()).
Msg("edge discovery: giving new address to connection")
return addr, nil
}
@@ -133,8 +136,9 @@ func (ed *Edge) AvailableAddrs() int {
func (ed *Edge) GiveBack(addr *allregions.EdgeAddr, hasConnectivityError bool) bool {
ed.Lock()
defer ed.Unlock()
log := ed.log.With().
IPAddr(LogFieldIPAddress, addr.UDP.IP).Logger()
log.Debug().Msgf("edgediscovery - GiveBack: Address now unused")
ed.log.Debug().
Int(management.EventTypeKey, int(management.Cloudflared)).
IPAddr(LogFieldIPAddress, addr.UDP.IP).
Msg("edge discovery: gave back address to the pool")
return ed.regions.GiveBack(addr, hasConnectivityError)
}
+1
View File
@@ -16,6 +16,7 @@ var (
FeatureSerializedHeaders,
FeatureDatagramV2,
FeatureQUICSupportEOF,
FeatureManagementLogs,
}
)
+5 -4
View File
@@ -13,6 +13,7 @@ require (
github.com/getsentry/raven-go v0.2.0
github.com/getsentry/sentry-go v0.16.0
github.com/go-chi/chi/v5 v5.0.8
github.com/go-jose/go-jose/v3 v3.0.0
github.com/gobwas/ws v1.0.4
github.com/golang-collections/collections v0.0.0-20130729185459-604e922904d3
github.com/google/gopacket v1.1.19
@@ -36,8 +37,8 @@ require (
go.opentelemetry.io/otel/trace v1.6.3
go.opentelemetry.io/proto/otlp v0.15.0
go.uber.org/automaxprocs v1.4.0
golang.org/x/crypto v0.5.0
golang.org/x/net v0.7.0
golang.org/x/crypto v0.8.0
golang.org/x/net v0.9.0
golang.org/x/sync v0.1.0
golang.org/x/sys v0.7.0
golang.org/x/term v0.7.0
@@ -96,8 +97,8 @@ require (
github.com/russross/blackfriday/v2 v2.1.0 // indirect
golang.org/x/mod v0.8.0 // indirect
golang.org/x/oauth2 v0.4.0 // indirect
golang.org/x/text v0.7.0 // indirect
golang.org/x/tools v0.1.12 // indirect
golang.org/x/text v0.9.0 // indirect
golang.org/x/tools v0.6.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20221202195650-67e5cbc046fd // indirect
google.golang.org/grpc v1.51.0 // indirect
+11 -8
View File
@@ -178,6 +178,8 @@ github.com/go-errors/errors v1.4.2 h1:J6MZopCL4uSllY1OfXM374weqZFFItUbrImctkmUxI
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1/go.mod h1:vR7hzQXu2zJy9AVAgeJqvqgH9Q5CA+iKCZ2gyEVpxRU=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20191125211704-12ad95a8df72/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4/go.mod h1:tQ2UAYgL5IevRw8kRxooKSPJfGvJ9fJQFa0TUsXzTg8=
github.com/go-jose/go-jose/v3 v3.0.0 h1:s6rrhirfEP/CGIoc6p+PZAeogN2SxKav6Wp7+dyMWVo=
github.com/go-jose/go-jose/v3 v3.0.0/go.mod h1:RNkWWRld676jZEYoV3+XK8L2ZnNSvIsxFMht0mSX+u8=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
@@ -542,12 +544,13 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk
golang.org/x/crypto v0.0.0-20190313024323-a1f597ede03a/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190911031432-227b76d455e7/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200221231518-2aa609cf4a9d/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.5.0 h1:U/0M97KRkSFvyD/3FSmdP5W5swImpNgle/EHFhOsQPE=
golang.org/x/crypto v0.5.0/go.mod h1:NK/OQwhpMQP3MwtdjgLlYHnH9ebylxKWv3e0fK+mkQU=
golang.org/x/crypto v0.8.0 h1:pd9TJtTueMTVQXzk8E2XESSMQDj/U7OUu0PqJqPXQjQ=
golang.org/x/crypto v0.8.0/go.mod h1:mRqEX+O9/h5TFCrQhkgjo2yKi0yYA+9ecGkdQoHrywE=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -640,8 +643,8 @@ golang.org/x/net v0.0.0-20220425223048-2871e0cb64e4/go.mod h1:CfG3xpIq0wQ8r1q4Su
golang.org/x/net v0.0.0-20220607020251-c690dde0001d/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.0.0-20220826154423-83b083e8dc8b/go.mod h1:YDH+HFinaLZZlnHAfSS6ZXJJ9M9t4Dl22yv3iI2vPwk=
golang.org/x/net v0.7.0 h1:rJrUqqhjsgNp7KqAIc25s9pZnjU7TUcSY7HcVZjdn1g=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.9.0 h1:aWJ/m6xSmxWBx+V0XRHTlrYrPG56jKsLdTFmsSsCzOM=
golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181017192945-9dcd33a902f4/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20181203162652-d668ce993890/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
@@ -777,8 +780,8 @@ golang.org/x/text v0.3.4/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.5/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0 h1:4BRB4x83lYWy72KwLD/qYDuTu7q9PjSagHvijDw7cLo=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0 h1:2sjJmO8cDvYveuX97RDLsxlyUxLl+GHoLxBiRdHllBE=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
@@ -839,8 +842,8 @@ golang.org/x/tools v0.1.3/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.4/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.6-0.20210726203631-07bc1bf47fb2/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk=
golang.org/x/tools v0.1.12 h1:VveCTK38A2rkS8ZqFY25HIDFscX5X9OoEhJd3quQmXU=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0 h1:BOw41kyTf3PuCW1pVQf8+Cyg8pMlkYB1oo9iJ6D/lKM=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+6 -2
View File
@@ -94,7 +94,7 @@ func ParseIngress(conf *config.Configuration) (Ingress, error) {
// ParseIngressFromConfigAndCLI will parse the configuration rules from config files for ingress
// rules and then attempt to parse CLI for ingress rules.
// Will always return at least one valid ingress rule. If none are provided by the user, the default
// will be to return 502 status code for all incoming requests.
// will be to return 503 status code for all incoming requests.
func ParseIngressFromConfigAndCLI(conf *config.Configuration, c *cli.Context, log *zerolog.Logger) (Ingress, error) {
// Attempt to parse ingress rules from configuration
ingressRules, err := ParseIngress(conf)
@@ -110,7 +110,11 @@ func ParseIngressFromConfigAndCLI(conf *config.Configuration, c *cli.Context, lo
// --bastion for ssh bastion service
ingressRules, err = parseCLIIngress(c, false)
if errors.Is(err, ErrNoIngressRulesCLI) {
log.Warn().Msgf(ErrNoIngressRulesCLI.Error())
// Only log a warning if the tunnel is not a remotely managed tunnel and the config
// will be loaded after connecting.
if !c.IsSet("token") {
log.Warn().Msgf(ErrNoIngressRulesCLI.Error())
}
return newDefaultOrigin(c, log), nil
}
if err != nil {
+113 -16
View File
@@ -48,7 +48,13 @@ type ClientEvent struct {
// Additional filters can be provided to augment the log events requested.
type EventStartStreaming struct {
ClientEvent
Filters []string `json:"filters"`
Filters *StreamingFilters `json:"filters,omitempty"`
}
type StreamingFilters struct {
Events []LogEventType `json:"events,omitempty"`
Level *LogLevel `json:"level,omitempty"`
Sampling float64 `json:"sampling,omitempty"`
}
// EventStopStreaming signifies that the client wishes to halt receiving log events.
@@ -59,21 +65,37 @@ type EventStopStreaming struct {
// EventLog is the event that the server sends to the client with the log events.
type EventLog struct {
ServerEvent
Logs []Log `json:"logs"`
Logs []*Log `json:"logs"`
}
// LogEventType is the way that logging messages are able to be filtered.
// Example: assigning LogEventType.Cloudflared to a zerolog event will allow the client to filter for only
// the Cloudflared-related events.
type LogEventType int
type LogEventType int8
const (
Cloudflared LogEventType = 0
HTTP LogEventType = 1
TCP LogEventType = 2
UDP LogEventType = 3
// Cloudflared events are signficant to cloudflared operations like connection state changes.
// Cloudflared is also the default event type for any events that haven't been separated into a proper event type.
Cloudflared LogEventType = iota
HTTP
TCP
UDP
)
func ParseLogEventType(s string) (LogEventType, bool) {
switch s {
case "cloudflared":
return Cloudflared, true
case "http":
return HTTP, true
case "tcp":
return TCP, true
case "udp":
return UDP, true
}
return -1, false
}
func (l LogEventType) String() string {
switch l {
case Cloudflared:
@@ -89,24 +111,99 @@ func (l LogEventType) String() string {
}
}
func (l LogEventType) MarshalJSON() ([]byte, error) {
return json.Marshal(l.String())
}
func (e *LogEventType) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.New("unable to unmarshal LogEventType string")
}
if event, ok := ParseLogEventType(s); ok {
*e = event
return nil
}
return errors.New("unable to unmarshal LogEventType")
}
// LogLevel corresponds to the zerolog logging levels
// "panic", "fatal", and "trace" are exempt from this list as they are rarely used and, at least
// the the first two are limited to failure conditions that lead to cloudflared shutting down.
type LogLevel string
type LogLevel int8
const (
Debug LogLevel = "debug"
Info LogLevel = "info"
Warn LogLevel = "warn"
Error LogLevel = "error"
Debug LogLevel = 0
Info LogLevel = 1
Warn LogLevel = 2
Error LogLevel = 3
)
func ParseLogLevel(l string) (LogLevel, bool) {
switch l {
case "debug":
return Debug, true
case "info":
return Info, true
case "warn":
return Warn, true
case "error":
return Error, true
}
return -1, false
}
func (l LogLevel) String() string {
switch l {
case Debug:
return "debug"
case Info:
return "info"
case Warn:
return "warn"
case Error:
return "error"
default:
return ""
}
}
func (l LogLevel) MarshalJSON() ([]byte, error) {
return json.Marshal(l.String())
}
func (l *LogLevel) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return errors.New("unable to unmarshal LogLevel string")
}
if level, ok := ParseLogLevel(s); ok {
*l = level
return nil
}
return fmt.Errorf("unable to unmarshal LogLevel")
}
const (
// TimeKey aligns with the zerolog.TimeFieldName
TimeKey = "time"
// LevelKey aligns with the zerolog.LevelFieldName
LevelKey = "level"
// LevelKey aligns with the zerolog.MessageFieldName
MessageKey = "message"
// EventTypeKey is the custom JSON key of the LogEventType in ZeroLogEvent
EventTypeKey = "event"
// FieldsKey is a custom JSON key to match and store every other key for a zerolog event
FieldsKey = "fields"
)
// Log is the basic structure of the events that are sent to the client.
type Log struct {
Event LogEventType `json:"event"`
Timestamp string `json:"timestamp"`
Level LogLevel `json:"level"`
Message string `json:"message"`
Time string `json:"time,omitempty"`
Level LogLevel `json:"level,omitempty"`
Message string `json:"message,omitempty"`
Event LogEventType `json:"event,omitempty"`
Fields map[string]interface{} `json:"fields,omitempty"`
}
// IntoClientEvent unmarshals the provided ClientEvent into the proper type.
+90 -11
View File
@@ -11,14 +11,93 @@ import (
"github.com/cloudflare/cloudflared/internal/test"
)
var (
debugLevel *LogLevel
infoLevel *LogLevel
warnLevel *LogLevel
errorLevel *LogLevel
)
func init() {
// created here because we can't do a reference to a const enum, i.e. &Info
debugLevel := new(LogLevel)
*debugLevel = Debug
infoLevel := new(LogLevel)
*infoLevel = Info
warnLevel := new(LogLevel)
*warnLevel = Warn
errorLevel := new(LogLevel)
*errorLevel = Error
}
func TestIntoClientEvent_StartStreaming(t *testing.T) {
event := ClientEvent{
Type: StartStreaming,
event: []byte(`{"type": "start_streaming"}`),
for _, test := range []struct {
name string
expected EventStartStreaming
}{
{
name: "no filters",
expected: EventStartStreaming{ClientEvent: ClientEvent{Type: StartStreaming}},
},
{
name: "level filter",
expected: EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
Filters: &StreamingFilters{
Level: infoLevel,
},
},
},
{
name: "events filter",
expected: EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
Filters: &StreamingFilters{
Events: []LogEventType{Cloudflared, HTTP},
},
},
},
{
name: "sampling filter",
expected: EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
Filters: &StreamingFilters{
Sampling: 0.5,
},
},
},
{
name: "level and events filters",
expected: EventStartStreaming{
ClientEvent: ClientEvent{Type: StartStreaming},
Filters: &StreamingFilters{
Level: infoLevel,
Events: []LogEventType{Cloudflared},
Sampling: 0.5,
},
},
},
} {
t.Run(test.name, func(t *testing.T) {
data, err := json.Marshal(test.expected)
require.NoError(t, err)
event := ClientEvent{}
err = json.Unmarshal(data, &event)
require.NoError(t, err)
event.event = data
ce, ok := IntoClientEvent[EventStartStreaming](&event, StartStreaming)
require.True(t, ok)
require.Equal(t, test.expected.ClientEvent, ce.ClientEvent)
if test.expected.Filters != nil {
f := ce.Filters
ef := test.expected.Filters
if ef.Level != nil {
require.Equal(t, *ef.Level, *f.Level)
}
require.ElementsMatch(t, ef.Events, f.Events)
}
})
}
ce, ok := IntoClientEvent[EventStartStreaming](&event, StartStreaming)
require.True(t, ok)
require.Equal(t, EventStartStreaming{ClientEvent: ClientEvent{Type: StartStreaming}}, *ce)
}
func TestIntoClientEvent_StopStreaming(t *testing.T) {
@@ -62,12 +141,12 @@ func TestIntoServerEvent_Invalid(t *testing.T) {
func TestReadServerEvent(t *testing.T) {
sentEvent := EventLog{
ServerEvent: ServerEvent{Type: Logs},
Logs: []Log{
Logs: []*Log{
{
Timestamp: time.Now().UTC().Format(time.RFC3339),
Event: HTTP,
Level: Info,
Message: "test",
Time: time.Now().UTC().Format(time.RFC3339),
Event: HTTP,
Level: Info,
Message: "test",
},
},
}
+98 -63
View File
@@ -11,24 +11,9 @@ import (
var json = jsoniter.ConfigFastest
const (
// Indicates how many log messages the listener will hold before dropping.
// Provides a throttling mechanism to drop latest messages if the sender
// can't keep up with the influx of log messages.
logWindow = 30
)
// ZeroLogEvent is the json structure that zerolog stores it's events as
type ZeroLogEvent struct {
Time string `json:"time,omitempty"`
Level LogLevel `json:"level,omitempty"`
Type LogEventType `json:"type,omitempty"`
Message string `json:"message,omitempty"`
}
// Logger manages the number of management streaming log sessions
type Logger struct {
sessions []*Session
sessions []*session
mu sync.RWMutex
// Unique logger that isn't a io.Writer of the list of zerolog writers. This helps prevent management log
@@ -48,35 +33,47 @@ func NewLogger() *Logger {
}
type LoggerListener interface {
Listen() *Session
Close(*Session)
// ActiveSession returns the first active session for the requested actor.
ActiveSession(actor) *session
// ActiveSession returns the count of active sessions.
ActiveSessions() int
// Listen appends the session to the list of sessions that receive log events.
Listen(*session)
// Remove a session from the available sessions that were receiving log events.
Remove(*session)
}
type Session struct {
// Buffered channel that holds the recent log events
listener chan *ZeroLogEvent
// Types of log events that this session will provide through the listener
filters []LogEventType
}
func newListener(size int) *Session {
return &Session{
listener: make(chan *ZeroLogEvent, size),
filters: []LogEventType{},
func (l *Logger) ActiveSession(actor actor) *session {
l.mu.RLock()
defer l.mu.RUnlock()
for _, session := range l.sessions {
if session.actor.ID == actor.ID && session.active.Load() {
return session
}
}
return nil
}
// Listen creates a new Session that will append filtered log events as they are created.
func (l *Logger) Listen() *Session {
func (l *Logger) ActiveSessions() int {
l.mu.RLock()
defer l.mu.RUnlock()
count := 0
for _, session := range l.sessions {
if session.active.Load() {
count += 1
}
}
return count
}
func (l *Logger) Listen(session *session) {
l.mu.Lock()
defer l.mu.Unlock()
listener := newListener(logWindow)
l.sessions = append(l.sessions, listener)
return listener
session.active.Store(true)
l.sessions = append(l.sessions, session)
}
// Close will remove a Session from the available sessions that were receiving log events.
func (l *Logger) Close(session *Session) {
func (l *Logger) Remove(session *session) {
l.mu.Lock()
defer l.mu.Unlock()
index := -1
@@ -104,35 +101,14 @@ func (l *Logger) Write(p []byte) (int, error) {
if len(l.sessions) == 0 {
return len(p), nil
}
var event ZeroLogEvent
iter := json.BorrowIterator(p)
defer json.ReturnIterator(iter)
iter.ReadVal(&event)
if iter.Error != nil {
l.Log.Debug().Msg("unable to unmarshal log event")
event, err := parseZerologEvent(p)
// drop event if unable to parse properly
if err != nil {
l.Log.Debug().Msg("unable to parse log event")
return len(p), nil
}
for _, listener := range l.sessions {
// no filters means all types are allowed
if len(listener.filters) != 0 {
valid := false
// make sure listener is subscribed to this event type
for _, t := range listener.filters {
if t == event.Type {
valid = true
break
}
}
if !valid {
continue
}
}
select {
case listener.listener <- &event:
default:
// buffer is full, discard
}
for _, session := range l.sessions {
session.Insert(event)
}
return len(p), nil
}
@@ -140,3 +116,62 @@ func (l *Logger) Write(p []byte) (int, error) {
func (l *Logger) WriteLevel(level zerolog.Level, p []byte) (n int, err error) {
return l.Write(p)
}
func parseZerologEvent(p []byte) (*Log, error) {
var fields map[string]interface{}
iter := json.BorrowIterator(p)
defer json.ReturnIterator(iter)
iter.ReadVal(&fields)
if iter.Error != nil {
return nil, iter.Error
}
logTime := time.Now().UTC().Format(zerolog.TimeFieldFormat)
if t, ok := fields[TimeKey]; ok {
if t, ok := t.(string); ok {
logTime = t
}
}
logLevel := Debug
// A zerolog Debug event can be created and then an error can be added
// via .Err(error), if so, we upgrade the level to error.
if _, hasError := fields["error"]; hasError {
logLevel = Error
} else {
if level, ok := fields[LevelKey]; ok {
if level, ok := level.(string); ok {
if logLevel, ok = ParseLogLevel(level); !ok {
logLevel = Debug
}
}
}
}
// Assume the event type is Cloudflared if unable to parse/find. This could be from log events that haven't
// yet been tagged with the appropriate EventType yet.
logEvent := Cloudflared
e := fields[EventTypeKey]
if e != nil {
if eventNumber, ok := e.(float64); ok {
logEvent = LogEventType(eventNumber)
}
}
logMessage := ""
if m, ok := fields[MessageKey]; ok {
if m, ok := m.(string); ok {
logMessage = m
}
}
event := Log{
Time: logTime,
Level: logLevel,
Event: logEvent,
Message: logMessage,
}
// Remove the keys that have top level keys on Log
delete(fields, TimeKey)
delete(fields, LevelKey)
delete(fields, EventTypeKey)
delete(fields, MessageKey)
// The rest of the keys go into the Fields
event.Fields = fields
return &event, nil
}
+93 -16
View File
@@ -1,10 +1,12 @@
package management
import (
"context"
"testing"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// No listening sessions will not write to the channel
@@ -19,15 +21,21 @@ func TestLoggerWrite_NoSessions(t *testing.T) {
func TestLoggerWrite_OneSession(t *testing.T) {
logger := NewLogger()
zlog := zerolog.New(logger).With().Timestamp().Logger().Level(zerolog.InfoLevel)
_, cancel := context.WithCancel(context.Background())
defer cancel()
session := logger.Listen()
defer logger.Close(session)
zlog.Info().Int("type", int(HTTP)).Msg("hello")
session := newSession(logWindow, actor{ID: actorID}, cancel)
logger.Listen(session)
defer logger.Remove(session)
assert.Equal(t, 1, logger.ActiveSessions())
assert.Equal(t, session, logger.ActiveSession(actor{ID: actorID}))
zlog.Info().Int(EventTypeKey, int(HTTP)).Msg("hello")
select {
case event := <-session.listener:
assert.NotEmpty(t, event.Time)
assert.Equal(t, "hello", event.Message)
assert.Equal(t, LogLevel("info"), event.Level)
assert.Equal(t, HTTP, event.Type)
assert.Equal(t, Info, event.Level)
assert.Equal(t, HTTP, event.Event)
default:
assert.Fail(t, "expected an event to be in the listener")
}
@@ -37,30 +45,40 @@ func TestLoggerWrite_OneSession(t *testing.T) {
func TestLoggerWrite_MultipleSessions(t *testing.T) {
logger := NewLogger()
zlog := zerolog.New(logger).With().Timestamp().Logger().Level(zerolog.InfoLevel)
_, cancel := context.WithCancel(context.Background())
defer cancel()
session1 := logger.Listen()
defer logger.Close(session1)
session2 := logger.Listen()
zlog.Info().Int("type", int(HTTP)).Msg("hello")
for _, session := range []*Session{session1, session2} {
session1 := newSession(logWindow, actor{}, cancel)
logger.Listen(session1)
defer logger.Remove(session1)
assert.Equal(t, 1, logger.ActiveSessions())
session2 := newSession(logWindow, actor{}, cancel)
logger.Listen(session2)
assert.Equal(t, 2, logger.ActiveSessions())
zlog.Info().Int(EventTypeKey, int(HTTP)).Msg("hello")
for _, session := range []*session{session1, session2} {
select {
case event := <-session.listener:
assert.NotEmpty(t, event.Time)
assert.Equal(t, "hello", event.Message)
assert.Equal(t, LogLevel("info"), event.Level)
assert.Equal(t, HTTP, event.Type)
assert.Equal(t, Info, event.Level)
assert.Equal(t, HTTP, event.Event)
default:
assert.Fail(t, "expected an event to be in the listener")
}
}
// Close session2 and make sure session1 still receives events
logger.Close(session2)
zlog.Info().Int("type", int(HTTP)).Msg("hello2")
logger.Remove(session2)
zlog.Info().Int(EventTypeKey, int(HTTP)).Msg("hello2")
select {
case event := <-session1.listener:
assert.NotEmpty(t, event.Time)
assert.Equal(t, "hello2", event.Message)
assert.Equal(t, LogLevel("info"), event.Level)
assert.Equal(t, HTTP, event.Type)
assert.Equal(t, Info, event.Level)
assert.Equal(t, HTTP, event.Event)
default:
assert.Fail(t, "expected an event to be in the listener")
}
@@ -73,3 +91,62 @@ func TestLoggerWrite_MultipleSessions(t *testing.T) {
// pass
}
}
type mockWriter struct {
event *Log
err error
}
func (m *mockWriter) Write(p []byte) (int, error) {
m.event, m.err = parseZerologEvent(p)
return len(p), nil
}
// Validate all event types are set properly
func TestParseZerologEvent_EventTypes(t *testing.T) {
writer := mockWriter{}
zlog := zerolog.New(&writer).With().Timestamp().Logger().Level(zerolog.InfoLevel)
for _, test := range []LogEventType{
Cloudflared,
HTTP,
TCP,
UDP,
} {
t.Run(test.String(), func(t *testing.T) {
defer func() { writer.err = nil }()
zlog.Info().Int(EventTypeKey, int(test)).Msg("test")
require.NoError(t, writer.err)
require.Equal(t, test, writer.event.Event)
})
}
// Invalid defaults to Cloudflared LogEventType
t.Run("invalid", func(t *testing.T) {
defer func() { writer.err = nil }()
zlog.Info().Str(EventTypeKey, "unknown").Msg("test")
require.NoError(t, writer.err)
require.Equal(t, Cloudflared, writer.event.Event)
})
}
// Validate top-level keys are removed from Fields
func TestParseZerologEvent_Fields(t *testing.T) {
writer := mockWriter{}
zlog := zerolog.New(&writer).With().Timestamp().Logger().Level(zerolog.InfoLevel)
zlog.Info().Int(EventTypeKey, int(Cloudflared)).Str("test", "test").Msg("test message")
require.NoError(t, writer.err)
event := writer.event
require.NotEmpty(t, event.Time)
require.Equal(t, Cloudflared, event.Event)
require.Equal(t, Info, event.Level)
require.Equal(t, "test message", event.Message)
// Make sure Fields doesn't have other set keys used in the Log struct
require.NotEmpty(t, event.Fields)
require.Equal(t, "test", event.Fields["test"])
require.NotContains(t, event.Fields, EventTypeKey)
require.NotContains(t, event.Fields, LevelKey)
require.NotContains(t, event.Fields, MessageKey)
require.NotContains(t, event.Fields, TimeKey)
}
+76
View File
@@ -0,0 +1,76 @@
package management
import (
"context"
"fmt"
"net/http"
)
type ctxKey int
const (
accessClaimsCtxKey ctxKey = iota
)
const (
connectorIDQuery = "connector_id"
accessTokenQuery = "access_token"
)
var (
errMissingAccessToken = managementError{Code: 1001, Message: "missing access_token query parameter"}
)
// HTTP middleware setting the parsed access_token claims in the request context
func ValidateAccessTokenQueryMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Validate access token
accessToken := r.URL.Query().Get("access_token")
if accessToken == "" {
writeHTTPErrorResponse(w, errMissingAccessToken)
return
}
token, err := parseToken(accessToken)
if err != nil {
writeHTTPErrorResponse(w, errMissingAccessToken)
return
}
r = r.WithContext(context.WithValue(r.Context(), accessClaimsCtxKey, token))
next.ServeHTTP(w, r)
})
}
// Middleware validation error struct for returning to the eyeball
type managementError struct {
Code int `json:"code,omitempty"`
Message string `json:"message,omitempty"`
}
func (m *managementError) Error() string {
return m.Message
}
// Middleware validation error HTTP response JSON for returning to the eyeball
type managementErrorResponse struct {
Success bool `json:"success,omitempty"`
Errors []managementError `json:"errors,omitempty"`
}
// writeErrorResponse will respond to the eyeball with basic HTTP JSON payloads with validation failure information
func writeHTTPErrorResponse(w http.ResponseWriter, errResp managementError) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusBadRequest)
err := json.NewEncoder(w).Encode(managementErrorResponse{
Success: false,
Errors: []managementError{errResp},
})
// we have already written the header, so write a basic error response if unable to encode the error
if err != nil {
// fallback to text message
http.Error(w, fmt.Sprintf(
"%d %s",
http.StatusBadRequest,
http.StatusText(http.StatusBadRequest),
), http.StatusBadRequest)
}
}
+71
View File
@@ -0,0 +1,71 @@
package management
import (
"io"
"net/http"
"net/http/httptest"
"testing"
"github.com/go-chi/chi/v5"
"github.com/stretchr/testify/assert"
)
func TestValidateAccessTokenQueryMiddleware(t *testing.T) {
r := chi.NewRouter()
r.Use(ValidateAccessTokenQueryMiddleware)
r.Get("/valid", func(w http.ResponseWriter, r *http.Request) {
claims, ok := r.Context().Value(accessClaimsCtxKey).(*managementTokenClaims)
assert.True(t, ok)
assert.True(t, claims.verify())
w.WriteHeader(http.StatusOK)
})
r.Get("/invalid", func(w http.ResponseWriter, r *http.Request) {
_, ok := r.Context().Value(accessClaimsCtxKey).(*managementTokenClaims)
assert.False(t, ok)
w.WriteHeader(http.StatusOK)
})
ts := httptest.NewServer(r)
defer ts.Close()
// valid: with access_token query param
path := "/valid?access_token=" + validToken
resp, _ := testRequest(t, ts, "GET", path, nil)
assert.Equal(t, http.StatusOK, resp.StatusCode)
// invalid: unset token
path = "/invalid"
resp, err := testRequest(t, ts, "GET", path, nil)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
assert.NotNil(t, err)
assert.Equal(t, errMissingAccessToken, err.Errors[0])
// invalid: invalid token
path = "/invalid?access_token=eyJ"
resp, err = testRequest(t, ts, "GET", path, nil)
assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
assert.NotNil(t, err)
assert.Equal(t, errMissingAccessToken, err.Errors[0])
}
func testRequest(t *testing.T, ts *httptest.Server, method, path string, body io.Reader) (*http.Response, *managementErrorResponse) {
req, err := http.NewRequest(method, ts.URL+path, body)
if err != nil {
t.Fatal(err)
return nil, nil
}
resp, err := ts.Client().Do(req)
if err != nil {
t.Fatal(err)
return nil, nil
}
var claims managementErrorResponse
err = json.NewDecoder(resp.Body).Decode(&claims)
if err != nil {
return resp, nil
}
defer resp.Body.Close()
return resp, &claims
}
+155 -39
View File
@@ -2,11 +2,15 @@ package management
import (
"context"
"fmt"
"net"
"net/http"
"os"
"sync"
"sync/atomic"
"time"
"github.com/go-chi/chi/v5"
"github.com/google/uuid"
"github.com/rs/zerolog"
"nhooyr.io/websocket"
)
@@ -19,18 +23,23 @@ const (
// value will return this error to incoming requests.
StatusSessionLimitExceeded websocket.StatusCode = 4002
reasonSessionLimitExceeded = "limit exceeded for streaming sessions"
// There is a limited idle time while not actively serving a session for a request before dropping the connection.
StatusIdleLimitExceeded websocket.StatusCode = 4003
reasonIdleLimitExceeded = "session was idle for too long"
)
type ManagementService struct {
// The management tunnel hostname
Hostname string
// Host details related configurations
serviceIP string
clientID uuid.UUID
label string
log *zerolog.Logger
router chi.Router
// streaming signifies if the service is already streaming logs. Helps limit the number of active users streaming logs
// from this cloudflared instance.
streaming atomic.Bool
// streamingMut is a lock to prevent concurrent requests to start streaming. Utilizing the atomic.Bool is not
// sufficient to complete this operation since many other checks during an incoming new request are needed
// to validate this before setting streaming to true.
@@ -38,16 +47,27 @@ type ManagementService struct {
logger LoggerListener
}
func New(managementHostname string, log *zerolog.Logger, logger LoggerListener) *ManagementService {
func New(managementHostname string,
serviceIP string,
clientID uuid.UUID,
label string,
log *zerolog.Logger,
logger LoggerListener,
) *ManagementService {
s := &ManagementService{
Hostname: managementHostname,
log: log,
logger: logger,
Hostname: managementHostname,
log: log,
logger: logger,
serviceIP: serviceIP,
clientID: clientID,
label: label,
}
r := chi.NewRouter()
r.Use(ValidateAccessTokenQueryMiddleware)
r.Get("/ping", ping)
r.Head("/ping", ping)
r.Get("/logs", s.logs)
r.Get("/host_details", s.getHostDetails)
s.router = r
return s
}
@@ -61,6 +81,53 @@ func ping(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(200)
}
// The response provided by the /host_details endpoint
type getHostDetailsResponse struct {
ClientID string `json:"connector_id"`
IP string `json:"ip,omitempty"`
HostName string `json:"hostname,omitempty"`
}
func (m *ManagementService) getHostDetails(w http.ResponseWriter, r *http.Request) {
var getHostDetailsResponse = getHostDetailsResponse{
ClientID: m.clientID.String(),
}
if ip, err := getPrivateIP(m.serviceIP); err == nil {
getHostDetailsResponse.IP = ip
}
getHostDetailsResponse.HostName = m.getLabel()
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(200)
json.NewEncoder(w).Encode(getHostDetailsResponse)
}
func (m *ManagementService) getLabel() string {
if m.label != "" {
return fmt.Sprintf("custom:%s", m.label)
}
// If no label is provided we return the system hostname. This is not
// a fqdn hostname.
hostname, err := os.Hostname()
if err != nil {
return "unknown"
}
return hostname
}
// Get preferred private ip of this machine
func getPrivateIP(addr string) (string, error) {
conn, err := net.DialTimeout("tcp", addr, 1*time.Second)
if err != nil {
return "", err
}
defer conn.Close()
localAddr := conn.LocalAddr().String()
host, _, err := net.SplitHostPort(localAddr)
return host, err
}
// readEvents will loop through all incoming websocket messages from a client and marshal them into the
// proper Event structure and pass through to the events channel. Any invalid messages sent will automatically
// terminate the connection.
@@ -86,22 +153,16 @@ func (m *ManagementService) readEvents(c *websocket.Conn, ctx context.Context, e
}
// streamLogs will begin the process of reading from the Session listener and write the log events to the client.
func (m *ManagementService) streamLogs(c *websocket.Conn, ctx context.Context, session *Session) {
defer m.logger.Close(session)
for m.streaming.Load() {
func (m *ManagementService) streamLogs(c *websocket.Conn, ctx context.Context, session *session) {
for session.Active() {
select {
case <-ctx.Done():
m.streaming.Store(false)
session.Stop()
return
case event := <-session.listener:
err := WriteEvent(c, ctx, &EventLog{
ServerEvent: ServerEvent{Type: Logs},
Logs: []Log{{
Event: Cloudflared,
Timestamp: event.Time,
Level: event.Level,
Message: event.Message,
}},
Logs: []*Log{event},
})
if err != nil {
// If the client (or the server) already closed the connection, don't attempt to close it again
@@ -110,7 +171,7 @@ func (m *ManagementService) streamLogs(c *websocket.Conn, ctx context.Context, s
m.log.Err(c.Close(websocket.StatusInternalError, err.Error())).Send()
}
// Any errors when writing the messages to the client will stop streaming and close the connection
m.streaming.Store(false)
session.Stop()
return
}
default:
@@ -119,28 +180,38 @@ func (m *ManagementService) streamLogs(c *websocket.Conn, ctx context.Context, s
}
}
// startStreaming will check the conditions of the request and begin streaming or close the connection for invalid
// requests.
func (m *ManagementService) startStreaming(c *websocket.Conn, ctx context.Context, event *ClientEvent) {
// canStartStream will check the conditions of the request and return if the session can begin streaming.
func (m *ManagementService) canStartStream(session *session) bool {
m.streamingMut.Lock()
defer m.streamingMut.Unlock()
// Limits to one user for streaming logs
if m.streaming.Load() {
m.log.Warn().
Msgf("Another management session request was attempted but one session already being served; there is a limit of streaming log sessions to reduce overall performance impact.")
m.log.Err(c.Close(StatusSessionLimitExceeded, reasonSessionLimitExceeded)).Send()
return
// Limits to one actor for streaming logs
if m.logger.ActiveSessions() > 0 {
// Allow the same user to preempt their existing session to disconnect their old session and start streaming
// with this new session instead.
if existingSession := m.logger.ActiveSession(session.actor); existingSession != nil {
m.log.Info().
Msgf("Another management session request for the same actor was requested; the other session will be disconnected to handle the new request.")
existingSession.Stop()
m.logger.Remove(existingSession)
existingSession.cancel()
} else {
m.log.Warn().
Msgf("Another management session request was attempted but one session already being served; there is a limit of streaming log sessions to reduce overall performance impact.")
return false
}
}
return true
}
// parseFilters will check the ClientEvent for start_streaming and assign filters if provided to the session
func (m *ManagementService) parseFilters(c *websocket.Conn, event *ClientEvent, session *session) bool {
// Expect the first incoming request
_, ok := IntoClientEvent[EventStartStreaming](event, StartStreaming)
startEvent, ok := IntoClientEvent[EventStartStreaming](event, StartStreaming)
if !ok {
m.log.Err(c.Close(StatusInvalidCommand, reasonInvalidCommand)).Msgf("expected start_streaming as first recieved event")
return
return false
}
m.streaming.Store(true)
listener := m.logger.Listen()
m.log.Debug().Msgf("Streaming logs")
go m.streamLogs(c, ctx, listener)
session.Filters(startEvent.Filters)
return true
}
// Management Streaming Logs accept handler
@@ -152,10 +223,32 @@ func (m *ManagementService) logs(w http.ResponseWriter, r *http.Request) {
}
// Make sure the connection is closed if other go routines fail to close the connection after completing.
defer c.Close(websocket.StatusInternalError, "")
ctx := r.Context()
ctx, cancel := context.WithCancel(r.Context())
defer cancel()
events := make(chan *ClientEvent)
go m.readEvents(c, ctx, events)
// Send a heartbeat ping to hold the connection open even if not streaming.
ping := time.NewTicker(15 * time.Second)
defer ping.Stop()
// Close the connection if no operation has occurred after the idle timeout. The timeout is halted
// when streaming logs is active.
idleTimeout := 5 * time.Minute
idle := time.NewTimer(idleTimeout)
defer idle.Stop()
// Fetch the claims from the request context to acquire the actor
claims, ok := ctx.Value(accessClaimsCtxKey).(*managementTokenClaims)
if !ok || claims == nil {
// Typically should never happen as it is provided in the context from the middleware
m.log.Err(c.Close(websocket.StatusInternalError, "missing access_token")).Send()
return
}
session := newSession(logWindow, claims.Actor, cancel)
defer m.logger.Remove(session)
for {
select {
case <-ctx.Done():
@@ -165,11 +258,29 @@ func (m *ManagementService) logs(w http.ResponseWriter, r *http.Request) {
case event := <-events:
switch event.Type {
case StartStreaming:
m.startStreaming(c, ctx, event)
idle.Stop()
// Expect the first incoming request
startEvent, ok := IntoClientEvent[EventStartStreaming](event, StartStreaming)
if !ok {
m.log.Warn().Msgf("expected start_streaming as first recieved event")
m.log.Err(c.Close(StatusInvalidCommand, reasonInvalidCommand)).Send()
return
}
// Make sure the session can start
if !m.canStartStream(session) {
m.log.Err(c.Close(StatusSessionLimitExceeded, reasonSessionLimitExceeded)).Send()
return
}
session.Filters(startEvent.Filters)
m.logger.Listen(session)
m.log.Debug().Msgf("Streaming logs")
go m.streamLogs(c, ctx, session)
continue
case StopStreaming:
// TODO: limit StopStreaming to only halt streaming for clients that are already streaming
m.streaming.Store(false)
idle.Reset(idleTimeout)
// Stop the current session for the current actor who requested it
session.Stop()
m.logger.Remove(session)
case UnknownClientEventType:
fallthrough
default:
@@ -181,6 +292,11 @@ func (m *ManagementService) logs(w http.ResponseWriter, r *http.Request) {
}
return
}
case <-ping.C:
go c.Ping(ctx)
case <-idle.C:
c.Close(StatusIdleLimitExceeded, reasonIdleLimitExceeded)
return
}
}
}
+65
View File
@@ -7,6 +7,7 @@ import (
"time"
"github.com/rs/zerolog"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"nhooyr.io/websocket"
@@ -59,3 +60,67 @@ func TestReadEventsLoop_ContextCancelled(t *testing.T) {
m.readEvents(server, ctx, events)
server.Close(websocket.StatusInternalError, "")
}
func TestCanStartStream_NoSessions(t *testing.T) {
m := ManagementService{
log: &noopLogger,
logger: &Logger{
Log: &noopLogger,
},
}
_, cancel := context.WithCancel(context.Background())
session := newSession(0, actor{}, cancel)
assert.True(t, m.canStartStream(session))
}
func TestCanStartStream_ExistingSessionDifferentActor(t *testing.T) {
m := ManagementService{
log: &noopLogger,
logger: &Logger{
Log: &noopLogger,
},
}
_, cancel := context.WithCancel(context.Background())
session1 := newSession(0, actor{ID: "test"}, cancel)
assert.True(t, m.canStartStream(session1))
m.logger.Listen(session1)
assert.True(t, session1.Active())
// Try another session
session2 := newSession(0, actor{ID: "test2"}, cancel)
assert.Equal(t, 1, m.logger.ActiveSessions())
assert.False(t, m.canStartStream(session2))
// Close session1
m.logger.Remove(session1)
assert.True(t, session1.Active()) // Remove doesn't stop a session
session1.Stop()
assert.False(t, session1.Active())
assert.Equal(t, 0, m.logger.ActiveSessions())
// Try session2 again
assert.True(t, m.canStartStream(session2))
}
func TestCanStartStream_ExistingSessionSameActor(t *testing.T) {
m := ManagementService{
log: &noopLogger,
logger: &Logger{
Log: &noopLogger,
},
}
actor := actor{ID: "test"}
_, cancel := context.WithCancel(context.Background())
session1 := newSession(0, actor, cancel)
assert.True(t, m.canStartStream(session1))
m.logger.Listen(session1)
assert.True(t, session1.Active())
// Try another session
session2 := newSession(0, actor, cancel)
assert.Equal(t, 1, m.logger.ActiveSessions())
assert.True(t, m.canStartStream(session2))
// session1 is removed and stopped
assert.Equal(t, 0, m.logger.ActiveSessions())
assert.False(t, session1.Active())
}
+120
View File
@@ -0,0 +1,120 @@
package management
import (
"context"
"math/rand"
"sync/atomic"
)
const (
// Indicates how many log messages the listener will hold before dropping.
// Provides a throttling mechanism to drop latest messages if the sender
// can't keep up with the influx of log messages.
logWindow = 30
)
// session captures a streaming logs session for a connection of an actor.
type session struct {
// Indicates if the session is streaming or not. Modifying this will affect the active session.
active atomic.Bool
// Allows the session to control the context of the underlying connection to close it out when done. Mostly
// used by the LoggerListener to close out and cleanup a session.
cancel context.CancelFunc
// Actor who started the session
actor actor
// Buffered channel that holds the recent log events
listener chan *Log
// Types of log events that this session will provide through the listener
filters *StreamingFilters
// Sampling of the log events this session will send (runs after all other filters if available)
sampler *sampler
}
// NewSession creates a new session.
func newSession(size int, actor actor, cancel context.CancelFunc) *session {
s := &session{
active: atomic.Bool{},
cancel: cancel,
actor: actor,
listener: make(chan *Log, size),
filters: &StreamingFilters{},
}
return s
}
// Filters assigns the StreamingFilters to the session
func (s *session) Filters(filters *StreamingFilters) {
if filters != nil {
s.filters = filters
sampling := filters.Sampling
// clamp the sampling values between 0 and 1
if sampling < 0 {
sampling = 0
}
if sampling > 1 {
sampling = 1
}
s.filters.Sampling = sampling
if sampling > 0 && sampling < 1 {
s.sampler = &sampler{
p: int(sampling * 100),
}
}
} else {
s.filters = &StreamingFilters{}
}
}
// Insert attempts to insert the log to the session. If the log event matches the provided session filters, it
// will be applied to the listener.
func (s *session) Insert(log *Log) {
// Level filters are optional
if s.filters.Level != nil {
if *s.filters.Level > log.Level {
return
}
}
// Event filters are optional
if len(s.filters.Events) != 0 && !contains(s.filters.Events, log.Event) {
return
}
// Sampling is also optional
if s.sampler != nil && !s.sampler.Sample() {
return
}
select {
case s.listener <- log:
default:
// buffer is full, discard
}
}
// Active returns if the session is active
func (s *session) Active() bool {
return s.active.Load()
}
// Stop will halt the session
func (s *session) Stop() {
s.active.Store(false)
}
func contains(array []LogEventType, t LogEventType) bool {
for _, v := range array {
if v == t {
return true
}
}
return false
}
// sampler will send approximately every p percentage log events out of 100.
type sampler struct {
p int
}
// Sample returns true if the event should be part of the sample, false if the event should be dropped.
func (s *sampler) Sample() bool {
return rand.Intn(100) <= s.p
}
+147
View File
@@ -0,0 +1,147 @@
package management
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Validate the active states of the session
func TestSession_ActiveControl(t *testing.T) {
_, cancel := context.WithCancel(context.Background())
defer cancel()
session := newSession(4, actor{}, cancel)
// session starts out not active
assert.False(t, session.Active())
session.active.Store(true)
assert.True(t, session.Active())
session.Stop()
assert.False(t, session.Active())
}
// Validate that the session filters events
func TestSession_Insert(t *testing.T) {
_, cancel := context.WithCancel(context.Background())
defer cancel()
infoLevel := new(LogLevel)
*infoLevel = Info
warnLevel := new(LogLevel)
*warnLevel = Warn
for _, test := range []struct {
name string
filters StreamingFilters
expectLog bool
}{
{
name: "none",
expectLog: true,
},
{
name: "level",
filters: StreamingFilters{
Level: infoLevel,
},
expectLog: true,
},
{
name: "filtered out level",
filters: StreamingFilters{
Level: warnLevel,
},
expectLog: false,
},
{
name: "events",
filters: StreamingFilters{
Events: []LogEventType{HTTP},
},
expectLog: true,
},
{
name: "filtered out event",
filters: StreamingFilters{
Events: []LogEventType{Cloudflared},
},
expectLog: false,
},
{
name: "sampling",
filters: StreamingFilters{
Sampling: 0.9999999,
},
expectLog: true,
},
{
name: "sampling (invalid negative)",
filters: StreamingFilters{
Sampling: -1.0,
},
expectLog: true,
},
{
name: "sampling (invalid too large)",
filters: StreamingFilters{
Sampling: 2.0,
},
expectLog: true,
},
{
name: "filter and event",
filters: StreamingFilters{
Level: infoLevel,
Events: []LogEventType{HTTP},
},
expectLog: true,
},
} {
t.Run(test.name, func(t *testing.T) {
session := newSession(4, actor{}, cancel)
session.Filters(&test.filters)
log := Log{
Time: time.Now().UTC().Format(time.RFC3339),
Event: HTTP,
Level: Info,
Message: "test",
}
session.Insert(&log)
select {
case <-session.listener:
require.True(t, test.expectLog)
default:
require.False(t, test.expectLog)
}
})
}
}
// Validate that the session has a max amount of events to hold
func TestSession_InsertOverflow(t *testing.T) {
_, cancel := context.WithCancel(context.Background())
defer cancel()
session := newSession(1, actor{}, cancel)
log := Log{
Time: time.Now().UTC().Format(time.RFC3339),
Event: HTTP,
Level: Info,
Message: "test",
}
// Insert 2 but only max channel size for 1
session.Insert(&log)
session.Insert(&log)
select {
case <-session.listener:
// pass
default:
require.Fail(t, "expected one log event")
}
// Second dequeue should fail
select {
case <-session.listener:
require.Fail(t, "expected no more remaining log events")
default:
// pass
}
}
+55
View File
@@ -0,0 +1,55 @@
package management
import (
"fmt"
"github.com/go-jose/go-jose/v3/jwt"
)
type managementTokenClaims struct {
Tunnel tunnel `json:"tun"`
Actor actor `json:"actor"`
}
// VerifyTunnel compares the tun claim isn't empty
func (c *managementTokenClaims) verify() bool {
return c.Tunnel.verify() && c.Actor.verify()
}
type tunnel struct {
ID string `json:"id"`
AccountTag string `json:"account_tag"`
}
// verify compares the tun claim isn't empty
func (t *tunnel) verify() bool {
return t.AccountTag != "" && t.ID != ""
}
type actor struct {
ID string `json:"id"`
Support bool `json:"support"`
}
// verify checks the ID claim isn't empty
func (t *actor) verify() bool {
return t.ID != ""
}
func parseToken(token string) (*managementTokenClaims, error) {
jwt, err := jwt.ParseSigned(token)
if err != nil {
return nil, fmt.Errorf("malformed jwt: %v", err)
}
var claims managementTokenClaims
// This is actually safe because we verify the token in the edge before it reaches cloudflared
err = jwt.UnsafeClaimsWithoutVerification(&claims)
if err != nil {
return nil, fmt.Errorf("malformed jwt: %v", err)
}
if !claims.verify() {
return nil, fmt.Errorf("invalid management token format provided")
}
return &claims, nil
}
+130
View File
@@ -0,0 +1,130 @@
package management
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"errors"
"testing"
"github.com/stretchr/testify/require"
"gopkg.in/square/go-jose.v2"
)
const (
validToken = "eyJ0eXAiOiJKV1QiLCJhbGciOiJFUzI1NiIsImtpZCI6IjEifQ.eyJ0dW4iOnsiaWQiOiI3YjA5ODE0OS01MWZlLTRlZTUtYTY4Ny0zZTM3NDQ2NmVmYzciLCJhY2NvdW50X3RhZyI6ImNkMzkxZTljMDYyNmE4Zjc2Y2IxZjY3MGY2NTkxYjA1In0sImFjdG9yIjp7ImlkIjoiZGNhcnJAY2xvdWRmbGFyZS5jb20iLCJzdXBwb3J0IjpmYWxzZX0sInJlcyI6WyJsb2dzIl0sImV4cCI6MTY3NzExNzY5NiwiaWF0IjoxNjc3MTE0MDk2LCJpc3MiOiJ0dW5uZWxzdG9yZSJ9.mKenOdOy3Xi4O-grldFnAAemdlE9WajEpTDC_FwezXQTstWiRTLwU65P5jt4vNsIiZA4OJRq7bH-QYID9wf9NA"
accountTag = "cd391e9c0626a8f76cb1f670f6591b05"
tunnelID = "7b098149-51fe-4ee5-a687-3e374466efc7"
actorID = "45d2751e-6b59-45a9-814d-f630786bd0cd"
)
type invalidManagementTokenClaims struct {
Invalid string `json:"invalid"`
}
func TestParseToken(t *testing.T) {
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
require.NoError(t, err)
for _, test := range []struct {
name string
claims any
err error
}{
{
name: "Valid",
claims: managementTokenClaims{
Tunnel: tunnel{
ID: tunnelID,
AccountTag: accountTag,
},
Actor: actor{
ID: actorID,
},
},
},
{
name: "Invalid claims",
claims: invalidManagementTokenClaims{Invalid: "invalid"},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Tunnel",
claims: managementTokenClaims{
Actor: actor{
ID: actorID,
},
},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Tunnel ID",
claims: managementTokenClaims{
Tunnel: tunnel{
AccountTag: accountTag,
},
Actor: actor{
ID: actorID,
},
},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Account Tag",
claims: managementTokenClaims{
Tunnel: tunnel{
ID: tunnelID,
},
Actor: actor{
ID: actorID,
},
},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Actor",
claims: managementTokenClaims{
Tunnel: tunnel{
ID: tunnelID,
AccountTag: accountTag,
},
},
err: errors.New("invalid management token format provided"),
},
{
name: "Missing Actor ID",
claims: managementTokenClaims{
Tunnel: tunnel{
ID: tunnelID,
},
Actor: actor{},
},
err: errors.New("invalid management token format provided"),
},
} {
t.Run(test.name, func(t *testing.T) {
jwt := signToken(t, test.claims, key)
claims, err := parseToken(jwt)
if test.err != nil {
require.EqualError(t, err, test.err.Error())
return
}
require.Nil(t, err)
require.Equal(t, test.claims, *claims)
})
}
}
func signToken(t *testing.T, token any, key *ecdsa.PrivateKey) string {
opts := (&jose.SignerOptions{}).WithType("JWT").WithHeader("kid", "1")
signer, err := jose.NewSigner(jose.SigningKey{Algorithm: jose.ES256, Key: key}, opts)
require.NoError(t, err)
payload, err := json.Marshal(token)
require.NoError(t, err)
jws, err := signer.Sign(payload)
require.NoError(t, err)
jwt, err := jws.CompactSerialize()
require.NoError(t, err)
return jwt
}
+2 -1
View File
@@ -14,6 +14,7 @@ import (
"time"
"github.com/gobwas/ws/wsutil"
"github.com/google/uuid"
gows "github.com/gorilla/websocket"
"github.com/rs/zerolog"
"github.com/stretchr/testify/require"
@@ -50,7 +51,7 @@ func TestUpdateConfiguration(t *testing.T) {
initConfig := &Config{
Ingress: &ingress.Ingress{},
}
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, []ingress.Rule{ingress.NewManagementRule(management.New("management.argotunnel.com", &testLogger, nil))}, &testLogger)
orchestrator, err := NewOrchestrator(context.Background(), initConfig, testTags, []ingress.Rule{ingress.NewManagementRule(management.New("management.argotunnel.com", "1.1.1.1:80", uuid.Nil, "", &testLogger, nil))}, &testLogger)
require.NoError(t, err)
initOriginProxy, err := orchestrator.GetOriginProxy()
require.NoError(t, err)
+44 -33
View File
@@ -16,6 +16,7 @@ import (
"github.com/cloudflare/cloudflared/cfio"
"github.com/cloudflare/cloudflared/connection"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/stream"
"github.com/cloudflare/cloudflared/tracing"
tunnelpogs "github.com/cloudflare/cloudflared/tunnelrpc/pogs"
@@ -25,10 +26,12 @@ const (
// TagHeaderNamePrefix indicates a Cloudflared Warp Tag prefix that gets appended for warp traffic stream headers.
TagHeaderNamePrefix = "Cf-Warp-Tag-"
LogFieldCFRay = "cfRay"
LogFieldLBProbe = "lbProbe"
LogFieldRule = "ingressRule"
LogFieldOriginService = "originService"
LogFieldFlowID = "flowID"
LogFieldConnIndex = "connIndex"
LogFieldDestAddr = "destAddr"
trailerHeaderName = "Trailer"
)
@@ -169,14 +172,24 @@ func (p *Proxy) ProxyTCP(
tracedCtx := tracing.NewTracedContext(serveCtx, req.CfTraceID, p.log)
p.log.Debug().Str(LogFieldFlowID, req.FlowID).Uint8(LogFieldConnIndex, req.ConnIndex).Msg("tcp proxy stream started")
p.log.Debug().
Int(management.EventTypeKey, int(management.TCP)).
Str(LogFieldFlowID, req.FlowID).
Str(LogFieldDestAddr, req.Dest).
Uint8(LogFieldConnIndex, req.ConnIndex).
Msg("tcp proxy stream started")
if err := p.proxyStream(tracedCtx, rwa, req.Dest, p.warpRouting.Proxy); err != nil {
p.logRequestError(err, req.CFRay, req.FlowID, "", ingress.ServiceWarpRouting)
return err
}
p.log.Debug().Str(LogFieldFlowID, req.FlowID).Uint8(LogFieldConnIndex, req.ConnIndex).Msg("tcp proxy stream finished successfully")
p.log.Debug().
Int(management.EventTypeKey, int(management.TCP)).
Str(LogFieldFlowID, req.FlowID).
Str(LogFieldDestAddr, req.Dest).
Uint8(LogFieldConnIndex, req.ConnIndex).
Msg("tcp proxy stream finished successfully")
return nil
}
@@ -339,7 +352,7 @@ func (p *Proxy) appendTagHeaders(r *http.Request) {
type logFields struct {
cfRay string
lbProbe bool
rule interface{}
rule int
flowID string
connIndex uint8
}
@@ -353,45 +366,41 @@ func copyTrailers(w connection.ResponseWriter, response *http.Response) {
}
func (p *Proxy) logRequest(r *http.Request, fields logFields) {
log := p.log.With().Int(management.EventTypeKey, int(management.HTTP)).Logger()
event := log.Debug()
if fields.cfRay != "" {
p.log.Debug().Msgf("CF-RAY: %s %s %s %s", fields.cfRay, r.Method, r.URL, r.Proto)
} else if fields.lbProbe {
p.log.Debug().Msgf("CF-RAY: %s Load Balancer health check %s %s %s", fields.cfRay, r.Method, r.URL, r.Proto)
} else {
p.log.Debug().Msgf("All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. %s %s %s ", r.Method, r.URL, r.Proto)
event = event.Str(LogFieldCFRay, fields.cfRay)
}
p.log.Debug().
Str("CF-RAY", fields.cfRay).
Str("Header", fmt.Sprintf("%+v", r.Header)).
if fields.lbProbe {
event = event.Bool(LogFieldLBProbe, fields.lbProbe)
}
if fields.cfRay == "" && !fields.lbProbe {
log.Debug().Msgf("All requests should have a CF-RAY header. Please open a support ticket with Cloudflare. %s %s %s ", r.Method, r.URL, r.Proto)
}
event.
Uint8(LogFieldConnIndex, fields.connIndex).
Str("host", r.Host).
Str("path", r.URL.Path).
Interface("rule", fields.rule).
Uint8(LogFieldConnIndex, fields.connIndex).
Msg("Inbound request")
if contentLen := r.ContentLength; contentLen == -1 {
p.log.Debug().Msgf("CF-RAY: %s Request Content length unknown", fields.cfRay)
} else {
p.log.Debug().Msgf("CF-RAY: %s Request content length %d", fields.cfRay, contentLen)
}
Interface(LogFieldRule, fields.rule).
Interface("headers", r.Header).
Int64("content-length", r.ContentLength).
Msgf("%s %s %s", r.Method, r.URL, r.Proto)
}
func (p *Proxy) logOriginResponse(resp *http.Response, fields logFields) {
responseByCode.WithLabelValues(strconv.Itoa(resp.StatusCode)).Inc()
event := p.log.Debug()
if fields.cfRay != "" {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("CF-RAY: %s Status: %s served by ingress %d", fields.cfRay, resp.Status, fields.rule)
} else if fields.lbProbe {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("Response to Load Balancer health check %s", resp.Status)
} else {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("Status: %s served by ingress %v", resp.Status, fields.rule)
event = event.Str(LogFieldCFRay, fields.cfRay)
}
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("CF-RAY: %s Response Headers %+v", fields.cfRay, resp.Header)
if contentLen := resp.ContentLength; contentLen == -1 {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("CF-RAY: %s Response content length unknown", fields.cfRay)
} else {
p.log.Debug().Uint8(LogFieldConnIndex, fields.connIndex).Msgf("CF-RAY: %s Response content length %d", fields.cfRay, contentLen)
if fields.lbProbe {
event = event.Bool(LogFieldLBProbe, fields.lbProbe)
}
event.
Int(management.EventTypeKey, int(management.HTTP)).
Uint8(LogFieldConnIndex, fields.connIndex).
Int64("content-length", resp.ContentLength).
Msgf("%s", resp.Status)
}
func (p *Proxy) logRequestError(err error, cfRay string, flowID string, rule, service string) {
@@ -401,7 +410,9 @@ func (p *Proxy) logRequestError(err error, cfRay string, flowID string, rule, se
log = log.Str(LogFieldCFRay, cfRay)
}
if flowID != "" {
log = log.Str(LogFieldFlowID, flowID)
log = log.Str(LogFieldFlowID, flowID).Int(management.EventTypeKey, int(management.TCP))
} else {
log = log.Int(management.EventTypeKey, int(management.HTTP))
}
if rule != "" {
log = log.Str(LogFieldRule, rule)
@@ -409,7 +420,7 @@ func (p *Proxy) logRequestError(err error, cfRay string, flowID string, rule, se
if service != "" {
log = log.Str(LogFieldOriginService, service)
}
log.Msg("")
log.Send()
}
func getDestFromRule(rule *ingress.Rule, req *http.Request) (string, error) {
+2
View File
@@ -22,6 +22,7 @@ import (
"github.com/cloudflare/cloudflared/features"
"github.com/cloudflare/cloudflared/h2mux"
"github.com/cloudflare/cloudflared/ingress"
"github.com/cloudflare/cloudflared/management"
"github.com/cloudflare/cloudflared/orchestration"
quicpogs "github.com/cloudflare/cloudflared/quic"
"github.com/cloudflare/cloudflared/retry"
@@ -238,6 +239,7 @@ func (e *EdgeTunnelServer) Serve(ctx context.Context, connIndex uint8, protocolF
}
logger := e.config.Log.With().
Int(management.EventTypeKey, int(management.Cloudflared)).
IPAddr(connection.LogFieldIPAddress, addr.UDP.IP).
Uint8(connection.LogFieldConnIndex, connIndex).
Logger()
+3 -3
View File
@@ -214,19 +214,19 @@ func getToken(appURL *url.URL, appInfo *AppInfo, useHostOnly bool, log *zerolog.
return appToken, nil
}
}
return getTokensFromEdge(appURL, appTokenPath, orgTokenPath, useHostOnly, log)
return getTokensFromEdge(appURL, appInfo.AppAUD, appTokenPath, orgTokenPath, useHostOnly, log)
}
// getTokensFromEdge will attempt to use the transfer service to retrieve an app and org token, save them to disk,
// and return the app token.
func getTokensFromEdge(appURL *url.URL, appTokenPath, orgTokenPath string, useHostOnly bool, log *zerolog.Logger) (string, error) {
func getTokensFromEdge(appURL *url.URL, appAUD, appTokenPath, orgTokenPath string, useHostOnly bool, log *zerolog.Logger) (string, error) {
// If no org token exists or if it couldn't be exchanged for an app token, then run the transfer service flow.
// this weird parameter is the resource name (token) and the key/value
// we want to send to the transfer service. the key is token and the value
// is blank (basically just the id generated in the transfer service)
resourceData, err := RunTransfer(appURL, keyName, keyName, "", true, useHostOnly, log)
resourceData, err := RunTransfer(appURL, appAUD, keyName, keyName, "", true, useHostOnly, log)
if err != nil {
return "", errors.Wrap(err, "failed to run transfer service")
}
+4 -3
View File
@@ -25,12 +25,12 @@ const (
// The "dance" we refer to is building a HTTP request, opening that in a browser waiting for
// the user to complete an action, while it long polls in the background waiting for an
// action to be completed to download the resource.
func RunTransfer(transferURL *url.URL, resourceName, key, value string, shouldEncrypt bool, useHostOnly bool, log *zerolog.Logger) ([]byte, error) {
func RunTransfer(transferURL *url.URL, appAUD, resourceName, key, value string, shouldEncrypt bool, useHostOnly bool, log *zerolog.Logger) ([]byte, error) {
encrypterClient, err := NewEncrypter("cloudflared_priv.pem", "cloudflared_pub.pem")
if err != nil {
return nil, err
}
requestURL, err := buildRequestURL(transferURL, key, value+encrypterClient.PublicKey(), shouldEncrypt, useHostOnly)
requestURL, err := buildRequestURL(transferURL, appAUD, key, value+encrypterClient.PublicKey(), shouldEncrypt, useHostOnly)
if err != nil {
return nil, err
}
@@ -76,9 +76,10 @@ func RunTransfer(transferURL *url.URL, resourceName, key, value string, shouldEn
// BuildRequestURL creates a request suitable for a resource transfer.
// it will return a constructed url based off the base url and query key/value provided.
// cli will build a url for cli transfer request.
func buildRequestURL(baseURL *url.URL, key, value string, cli, useHostOnly bool) (string, error) {
func buildRequestURL(baseURL *url.URL, appAUD string, key, value string, cli, useHostOnly bool) (string, error) {
q := baseURL.Query()
q.Set(key, value)
q.Set("aud", appAUD)
baseURL.RawQuery = q.Encode()
if useHostOnly {
baseURL.Path = ""
+3 -3
View File
@@ -24,17 +24,17 @@ func NewTransportLogger(log *zerolog.Logger, t rpc.Transport) rpc.Transport {
}
func (t *transport) SendMessage(ctx context.Context, msg rpccapnp.Message) error {
t.log.Debug().Msgf("rpcconnect: tx %s", formatMsg(msg))
t.log.Trace().Msgf("rpc tx: %s", formatMsg(msg))
return t.Transport.SendMessage(ctx, msg)
}
func (t *transport) RecvMessage(ctx context.Context) (rpccapnp.Message, error) {
msg, err := t.Transport.RecvMessage(ctx)
if err != nil {
t.log.Debug().Msgf("rpcconnect: rx error: %s", err)
t.log.Debug().Msgf("rpc rx error: %s", err)
return msg, err
}
t.log.Debug().Msgf("rpcconnect: rx %s", formatMsg(msg))
t.log.Trace().Msgf("rpc rx: %s", formatMsg(msg))
return msg, nil
}
+2
View File
@@ -0,0 +1,2 @@
jose-util/jose-util
jose-util.t.err
+53
View File
@@ -0,0 +1,53 @@
# https://github.com/golangci/golangci-lint
run:
skip-files:
- doc_test.go
modules-download-mode: readonly
linters:
enable-all: true
disable:
- gochecknoglobals
- goconst
- lll
- maligned
- nakedret
- scopelint
- unparam
- funlen # added in 1.18 (requires go-jose changes before it can be enabled)
linters-settings:
gocyclo:
min-complexity: 35
issues:
exclude-rules:
- text: "don't use ALL_CAPS in Go names"
linters:
- golint
- text: "hardcoded credentials"
linters:
- gosec
- text: "weak cryptographic primitive"
linters:
- gosec
- path: json/
linters:
- dupl
- errcheck
- gocritic
- gocyclo
- golint
- govet
- ineffassign
- staticcheck
- structcheck
- stylecheck
- unused
- path: _test\.go
linters:
- scopelint
- path: jwk.go
linters:
- gocyclo
+33
View File
@@ -0,0 +1,33 @@
language: go
matrix:
fast_finish: true
allow_failures:
- go: tip
go:
- "1.13.x"
- "1.14.x"
- tip
before_script:
- export PATH=$HOME/.local/bin:$PATH
before_install:
- go get -u github.com/mattn/goveralls github.com/wadey/gocovmerge
- curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin v1.18.0
- pip install cram --user
script:
- go test -v -covermode=count -coverprofile=profile.cov .
- go test -v -covermode=count -coverprofile=cryptosigner/profile.cov ./cryptosigner
- go test -v -covermode=count -coverprofile=cipher/profile.cov ./cipher
- go test -v -covermode=count -coverprofile=jwt/profile.cov ./jwt
- go test -v ./json # no coverage for forked encoding/json package
- golangci-lint run
- cd jose-util && go build && PATH=$PWD:$PATH cram -v jose-util.t # cram tests jose-util
- cd ..
after_success:
- gocovmerge *.cov */*.cov > merged.coverprofile
- goveralls -coverprofile merged.coverprofile -service=travis-ci
+10
View File
@@ -0,0 +1,10 @@
Serious about security
======================
Square recognizes the important contributions the security research community
can make. We therefore encourage reporting security issues with the code
contained in this repository.
If you believe you have discovered a security vulnerability, please follow the
guidelines at <https://bugcrowd.com/squareopensource>.
+15
View File
@@ -0,0 +1,15 @@
# Contributing
If you would like to contribute code to go-jose you can do so through GitHub by
forking the repository and sending a pull request.
When submitting code, please make every effort to follow existing conventions
and style in order to keep the code as readable as possible. Please also make
sure all tests pass by running `go test`, and format your code with `go fmt`.
We also recommend using `golint` and `errcheck`.
Before your code can be accepted into the project you must also sign the
Individual Contributor License Agreement. We use [cla-assistant.io][1] and you
will be prompted to sign once a pull request is opened.
[1]: https://cla-assistant.io/
+202
View File
@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+122
View File
@@ -0,0 +1,122 @@
# Go JOSE
[![godoc](http://img.shields.io/badge/godoc-jose_package-blue.svg?style=flat)](https://godoc.org/gopkg.in/go-jose/go-jose.v2)
[![godoc](http://img.shields.io/badge/godoc-jwt_package-blue.svg?style=flat)](https://godoc.org/gopkg.in/go-jose/go-jose.v2/jwt)
[![license](http://img.shields.io/badge/license-apache_2.0-blue.svg?style=flat)](https://raw.githubusercontent.com/go-jose/go-jose/master/LICENSE)
[![build](https://travis-ci.org/go-jose/go-jose.svg?branch=master)](https://travis-ci.org/go-jose/go-jose)
[![coverage](https://coveralls.io/repos/github/go-jose/go-jose/badge.svg?branch=master)](https://coveralls.io/r/go-jose/go-jose)
Package jose aims to provide an implementation of the Javascript Object Signing
and Encryption set of standards. This includes support for JSON Web Encryption,
JSON Web Signature, and JSON Web Token standards.
**Disclaimer**: This library contains encryption software that is subject to
the U.S. Export Administration Regulations. You may not export, re-export,
transfer or download this code or any part of it in violation of any United
States law, directive or regulation. In particular this software may not be
exported or re-exported in any form or on any media to Iran, North Sudan,
Syria, Cuba, or North Korea, or to denied persons or entities mentioned on any
US maintained blocked list.
## Overview
The implementation follows the
[JSON Web Encryption](http://dx.doi.org/10.17487/RFC7516) (RFC 7516),
[JSON Web Signature](http://dx.doi.org/10.17487/RFC7515) (RFC 7515), and
[JSON Web Token](http://dx.doi.org/10.17487/RFC7519) (RFC 7519) specifications.
Tables of supported algorithms are shown below. The library supports both
the compact and JWS/JWE JSON Serialization formats, and has optional support for
multiple recipients. It also comes with a small command-line utility
([`jose-util`](https://github.com/go-jose/go-jose/tree/master/jose-util))
for dealing with JOSE messages in a shell.
**Note**: We use a forked version of the `encoding/json` package from the Go
standard library which uses case-sensitive matching for member names (instead
of [case-insensitive matching](https://www.ietf.org/mail-archive/web/json/current/msg03763.html)).
This is to avoid differences in interpretation of messages between go-jose and
libraries in other languages.
### Versions
[Version 2](https://gopkg.in/go-jose/go-jose.v2)
([branch](https://github.com/go-jose/go-jose/tree/v2),
[doc](https://godoc.org/gopkg.in/go-jose/go-jose.v2)) is the current stable version:
import "gopkg.in/go-jose/go-jose.v2"
[Version 3](https://github.com/go-jose/go-jose)
([branch](https://github.com/go-jose/go-jose/tree/master),
[doc](https://godoc.org/github.com/go-jose/go-jose)) is the under development/unstable version (not released yet):
import "github.com/go-jose/go-jose/v3"
All new feature development takes place on the `master` branch, which we are
preparing to release as version 3 soon. Version 2 will continue to receive
critical bug and security fixes. Note that starting with version 3 we are
using Go modules for versioning instead of `gopkg.in` as before. Version 3 also will require Go version 1.13 or higher.
Version 1 (on the `v1` branch) is frozen and not supported anymore.
### Supported algorithms
See below for a table of supported algorithms. Algorithm identifiers match
the names in the [JSON Web Algorithms](http://dx.doi.org/10.17487/RFC7518)
standard where possible. The Godoc reference has a list of constants.
Key encryption | Algorithm identifier(s)
:------------------------- | :------------------------------
RSA-PKCS#1v1.5 | RSA1_5
RSA-OAEP | RSA-OAEP, RSA-OAEP-256
AES key wrap | A128KW, A192KW, A256KW
AES-GCM key wrap | A128GCMKW, A192GCMKW, A256GCMKW
ECDH-ES + AES key wrap | ECDH-ES+A128KW, ECDH-ES+A192KW, ECDH-ES+A256KW
ECDH-ES (direct) | ECDH-ES<sup>1</sup>
Direct encryption | dir<sup>1</sup>
<sup>1. Not supported in multi-recipient mode</sup>
Signing / MAC | Algorithm identifier(s)
:------------------------- | :------------------------------
RSASSA-PKCS#1v1.5 | RS256, RS384, RS512
RSASSA-PSS | PS256, PS384, PS512
HMAC | HS256, HS384, HS512
ECDSA | ES256, ES384, ES512
Ed25519 | EdDSA<sup>2</sup>
<sup>2. Only available in version 2 of the package</sup>
Content encryption | Algorithm identifier(s)
:------------------------- | :------------------------------
AES-CBC+HMAC | A128CBC-HS256, A192CBC-HS384, A256CBC-HS512
AES-GCM | A128GCM, A192GCM, A256GCM
Compression | Algorithm identifiers(s)
:------------------------- | -------------------------------
DEFLATE (RFC 1951) | DEF
### Supported key types
See below for a table of supported key types. These are understood by the
library, and can be passed to corresponding functions such as `NewEncrypter` or
`NewSigner`. Each of these keys can also be wrapped in a JWK if desired, which
allows attaching a key id.
Algorithm(s) | Corresponding types
:------------------------- | -------------------------------
RSA | *[rsa.PublicKey](http://golang.org/pkg/crypto/rsa/#PublicKey), *[rsa.PrivateKey](http://golang.org/pkg/crypto/rsa/#PrivateKey)
ECDH, ECDSA | *[ecdsa.PublicKey](http://golang.org/pkg/crypto/ecdsa/#PublicKey), *[ecdsa.PrivateKey](http://golang.org/pkg/crypto/ecdsa/#PrivateKey)
EdDSA<sup>1</sup> | [ed25519.PublicKey](https://godoc.org/pkg/crypto/ed25519#PublicKey), [ed25519.PrivateKey](https://godoc.org/pkg/crypto/ed25519#PrivateKey)
AES, HMAC | []byte
<sup>1. Only available in version 2 or later of the package</sup>
## Examples
[![godoc](http://img.shields.io/badge/godoc-jose_package-blue.svg?style=flat)](https://godoc.org/gopkg.in/go-jose/go-jose.v2)
[![godoc](http://img.shields.io/badge/godoc-jwt_package-blue.svg?style=flat)](https://godoc.org/gopkg.in/go-jose/go-jose.v2/jwt)
Examples can be found in the Godoc
reference for this package. The
[`jose-util`](https://github.com/go-jose/go-jose/tree/master/jose-util)
subdirectory also contains a small command-line utility which might be useful
as an example as well.
+592
View File
@@ -0,0 +1,592 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jose
import (
"crypto"
"crypto/aes"
"crypto/ecdsa"
"crypto/ed25519"
"crypto/rand"
"crypto/rsa"
"crypto/sha1"
"crypto/sha256"
"errors"
"fmt"
"math/big"
josecipher "github.com/go-jose/go-jose/v3/cipher"
"github.com/go-jose/go-jose/v3/json"
)
// A generic RSA-based encrypter/verifier
type rsaEncrypterVerifier struct {
publicKey *rsa.PublicKey
}
// A generic RSA-based decrypter/signer
type rsaDecrypterSigner struct {
privateKey *rsa.PrivateKey
}
// A generic EC-based encrypter/verifier
type ecEncrypterVerifier struct {
publicKey *ecdsa.PublicKey
}
type edEncrypterVerifier struct {
publicKey ed25519.PublicKey
}
// A key generator for ECDH-ES
type ecKeyGenerator struct {
size int
algID string
publicKey *ecdsa.PublicKey
}
// A generic EC-based decrypter/signer
type ecDecrypterSigner struct {
privateKey *ecdsa.PrivateKey
}
type edDecrypterSigner struct {
privateKey ed25519.PrivateKey
}
// newRSARecipient creates recipientKeyInfo based on the given key.
func newRSARecipient(keyAlg KeyAlgorithm, publicKey *rsa.PublicKey) (recipientKeyInfo, error) {
// Verify that key management algorithm is supported by this encrypter
switch keyAlg {
case RSA1_5, RSA_OAEP, RSA_OAEP_256:
default:
return recipientKeyInfo{}, ErrUnsupportedAlgorithm
}
if publicKey == nil {
return recipientKeyInfo{}, errors.New("invalid public key")
}
return recipientKeyInfo{
keyAlg: keyAlg,
keyEncrypter: &rsaEncrypterVerifier{
publicKey: publicKey,
},
}, nil
}
// newRSASigner creates a recipientSigInfo based on the given key.
func newRSASigner(sigAlg SignatureAlgorithm, privateKey *rsa.PrivateKey) (recipientSigInfo, error) {
// Verify that key management algorithm is supported by this encrypter
switch sigAlg {
case RS256, RS384, RS512, PS256, PS384, PS512:
default:
return recipientSigInfo{}, ErrUnsupportedAlgorithm
}
if privateKey == nil {
return recipientSigInfo{}, errors.New("invalid private key")
}
return recipientSigInfo{
sigAlg: sigAlg,
publicKey: staticPublicKey(&JSONWebKey{
Key: privateKey.Public(),
}),
signer: &rsaDecrypterSigner{
privateKey: privateKey,
},
}, nil
}
func newEd25519Signer(sigAlg SignatureAlgorithm, privateKey ed25519.PrivateKey) (recipientSigInfo, error) {
if sigAlg != EdDSA {
return recipientSigInfo{}, ErrUnsupportedAlgorithm
}
if privateKey == nil {
return recipientSigInfo{}, errors.New("invalid private key")
}
return recipientSigInfo{
sigAlg: sigAlg,
publicKey: staticPublicKey(&JSONWebKey{
Key: privateKey.Public(),
}),
signer: &edDecrypterSigner{
privateKey: privateKey,
},
}, nil
}
// newECDHRecipient creates recipientKeyInfo based on the given key.
func newECDHRecipient(keyAlg KeyAlgorithm, publicKey *ecdsa.PublicKey) (recipientKeyInfo, error) {
// Verify that key management algorithm is supported by this encrypter
switch keyAlg {
case ECDH_ES, ECDH_ES_A128KW, ECDH_ES_A192KW, ECDH_ES_A256KW:
default:
return recipientKeyInfo{}, ErrUnsupportedAlgorithm
}
if publicKey == nil || !publicKey.Curve.IsOnCurve(publicKey.X, publicKey.Y) {
return recipientKeyInfo{}, errors.New("invalid public key")
}
return recipientKeyInfo{
keyAlg: keyAlg,
keyEncrypter: &ecEncrypterVerifier{
publicKey: publicKey,
},
}, nil
}
// newECDSASigner creates a recipientSigInfo based on the given key.
func newECDSASigner(sigAlg SignatureAlgorithm, privateKey *ecdsa.PrivateKey) (recipientSigInfo, error) {
// Verify that key management algorithm is supported by this encrypter
switch sigAlg {
case ES256, ES384, ES512:
default:
return recipientSigInfo{}, ErrUnsupportedAlgorithm
}
if privateKey == nil {
return recipientSigInfo{}, errors.New("invalid private key")
}
return recipientSigInfo{
sigAlg: sigAlg,
publicKey: staticPublicKey(&JSONWebKey{
Key: privateKey.Public(),
}),
signer: &ecDecrypterSigner{
privateKey: privateKey,
},
}, nil
}
// Encrypt the given payload and update the object.
func (ctx rsaEncrypterVerifier) encryptKey(cek []byte, alg KeyAlgorithm) (recipientInfo, error) {
encryptedKey, err := ctx.encrypt(cek, alg)
if err != nil {
return recipientInfo{}, err
}
return recipientInfo{
encryptedKey: encryptedKey,
header: &rawHeader{},
}, nil
}
// Encrypt the given payload. Based on the key encryption algorithm,
// this will either use RSA-PKCS1v1.5 or RSA-OAEP (with SHA-1 or SHA-256).
func (ctx rsaEncrypterVerifier) encrypt(cek []byte, alg KeyAlgorithm) ([]byte, error) {
switch alg {
case RSA1_5:
return rsa.EncryptPKCS1v15(RandReader, ctx.publicKey, cek)
case RSA_OAEP:
return rsa.EncryptOAEP(sha1.New(), RandReader, ctx.publicKey, cek, []byte{})
case RSA_OAEP_256:
return rsa.EncryptOAEP(sha256.New(), RandReader, ctx.publicKey, cek, []byte{})
}
return nil, ErrUnsupportedAlgorithm
}
// Decrypt the given payload and return the content encryption key.
func (ctx rsaDecrypterSigner) decryptKey(headers rawHeader, recipient *recipientInfo, generator keyGenerator) ([]byte, error) {
return ctx.decrypt(recipient.encryptedKey, headers.getAlgorithm(), generator)
}
// Decrypt the given payload. Based on the key encryption algorithm,
// this will either use RSA-PKCS1v1.5 or RSA-OAEP (with SHA-1 or SHA-256).
func (ctx rsaDecrypterSigner) decrypt(jek []byte, alg KeyAlgorithm, generator keyGenerator) ([]byte, error) {
// Note: The random reader on decrypt operations is only used for blinding,
// so stubbing is meanlingless (hence the direct use of rand.Reader).
switch alg {
case RSA1_5:
defer func() {
// DecryptPKCS1v15SessionKey sometimes panics on an invalid payload
// because of an index out of bounds error, which we want to ignore.
// This has been fixed in Go 1.3.1 (released 2014/08/13), the recover()
// only exists for preventing crashes with unpatched versions.
// See: https://groups.google.com/forum/#!topic/golang-dev/7ihX6Y6kx9k
// See: https://code.google.com/p/go/source/detail?r=58ee390ff31602edb66af41ed10901ec95904d33
_ = recover()
}()
// Perform some input validation.
keyBytes := ctx.privateKey.PublicKey.N.BitLen() / 8
if keyBytes != len(jek) {
// Input size is incorrect, the encrypted payload should always match
// the size of the public modulus (e.g. using a 2048 bit key will
// produce 256 bytes of output). Reject this since it's invalid input.
return nil, ErrCryptoFailure
}
cek, _, err := generator.genKey()
if err != nil {
return nil, ErrCryptoFailure
}
// When decrypting an RSA-PKCS1v1.5 payload, we must take precautions to
// prevent chosen-ciphertext attacks as described in RFC 3218, "Preventing
// the Million Message Attack on Cryptographic Message Syntax". We are
// therefore deliberately ignoring errors here.
_ = rsa.DecryptPKCS1v15SessionKey(rand.Reader, ctx.privateKey, jek, cek)
return cek, nil
case RSA_OAEP:
// Use rand.Reader for RSA blinding
return rsa.DecryptOAEP(sha1.New(), rand.Reader, ctx.privateKey, jek, []byte{})
case RSA_OAEP_256:
// Use rand.Reader for RSA blinding
return rsa.DecryptOAEP(sha256.New(), rand.Reader, ctx.privateKey, jek, []byte{})
}
return nil, ErrUnsupportedAlgorithm
}
// Sign the given payload
func (ctx rsaDecrypterSigner) signPayload(payload []byte, alg SignatureAlgorithm) (Signature, error) {
var hash crypto.Hash
switch alg {
case RS256, PS256:
hash = crypto.SHA256
case RS384, PS384:
hash = crypto.SHA384
case RS512, PS512:
hash = crypto.SHA512
default:
return Signature{}, ErrUnsupportedAlgorithm
}
hasher := hash.New()
// According to documentation, Write() on hash never fails
_, _ = hasher.Write(payload)
hashed := hasher.Sum(nil)
var out []byte
var err error
switch alg {
case RS256, RS384, RS512:
out, err = rsa.SignPKCS1v15(RandReader, ctx.privateKey, hash, hashed)
case PS256, PS384, PS512:
out, err = rsa.SignPSS(RandReader, ctx.privateKey, hash, hashed, &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthEqualsHash,
})
}
if err != nil {
return Signature{}, err
}
return Signature{
Signature: out,
protected: &rawHeader{},
}, nil
}
// Verify the given payload
func (ctx rsaEncrypterVerifier) verifyPayload(payload []byte, signature []byte, alg SignatureAlgorithm) error {
var hash crypto.Hash
switch alg {
case RS256, PS256:
hash = crypto.SHA256
case RS384, PS384:
hash = crypto.SHA384
case RS512, PS512:
hash = crypto.SHA512
default:
return ErrUnsupportedAlgorithm
}
hasher := hash.New()
// According to documentation, Write() on hash never fails
_, _ = hasher.Write(payload)
hashed := hasher.Sum(nil)
switch alg {
case RS256, RS384, RS512:
return rsa.VerifyPKCS1v15(ctx.publicKey, hash, hashed, signature)
case PS256, PS384, PS512:
return rsa.VerifyPSS(ctx.publicKey, hash, hashed, signature, nil)
}
return ErrUnsupportedAlgorithm
}
// Encrypt the given payload and update the object.
func (ctx ecEncrypterVerifier) encryptKey(cek []byte, alg KeyAlgorithm) (recipientInfo, error) {
switch alg {
case ECDH_ES:
// ECDH-ES mode doesn't wrap a key, the shared secret is used directly as the key.
return recipientInfo{
header: &rawHeader{},
}, nil
case ECDH_ES_A128KW, ECDH_ES_A192KW, ECDH_ES_A256KW:
default:
return recipientInfo{}, ErrUnsupportedAlgorithm
}
generator := ecKeyGenerator{
algID: string(alg),
publicKey: ctx.publicKey,
}
switch alg {
case ECDH_ES_A128KW:
generator.size = 16
case ECDH_ES_A192KW:
generator.size = 24
case ECDH_ES_A256KW:
generator.size = 32
}
kek, header, err := generator.genKey()
if err != nil {
return recipientInfo{}, err
}
block, err := aes.NewCipher(kek)
if err != nil {
return recipientInfo{}, err
}
jek, err := josecipher.KeyWrap(block, cek)
if err != nil {
return recipientInfo{}, err
}
return recipientInfo{
encryptedKey: jek,
header: &header,
}, nil
}
// Get key size for EC key generator
func (ctx ecKeyGenerator) keySize() int {
return ctx.size
}
// Get a content encryption key for ECDH-ES
func (ctx ecKeyGenerator) genKey() ([]byte, rawHeader, error) {
priv, err := ecdsa.GenerateKey(ctx.publicKey.Curve, RandReader)
if err != nil {
return nil, rawHeader{}, err
}
out := josecipher.DeriveECDHES(ctx.algID, []byte{}, []byte{}, priv, ctx.publicKey, ctx.size)
b, err := json.Marshal(&JSONWebKey{
Key: &priv.PublicKey,
})
if err != nil {
return nil, nil, err
}
headers := rawHeader{
headerEPK: makeRawMessage(b),
}
return out, headers, nil
}
// Decrypt the given payload and return the content encryption key.
func (ctx ecDecrypterSigner) decryptKey(headers rawHeader, recipient *recipientInfo, generator keyGenerator) ([]byte, error) {
epk, err := headers.getEPK()
if err != nil {
return nil, errors.New("go-jose/go-jose: invalid epk header")
}
if epk == nil {
return nil, errors.New("go-jose/go-jose: missing epk header")
}
publicKey, ok := epk.Key.(*ecdsa.PublicKey)
if publicKey == nil || !ok {
return nil, errors.New("go-jose/go-jose: invalid epk header")
}
if !ctx.privateKey.Curve.IsOnCurve(publicKey.X, publicKey.Y) {
return nil, errors.New("go-jose/go-jose: invalid public key in epk header")
}
apuData, err := headers.getAPU()
if err != nil {
return nil, errors.New("go-jose/go-jose: invalid apu header")
}
apvData, err := headers.getAPV()
if err != nil {
return nil, errors.New("go-jose/go-jose: invalid apv header")
}
deriveKey := func(algID string, size int) []byte {
return josecipher.DeriveECDHES(algID, apuData.bytes(), apvData.bytes(), ctx.privateKey, publicKey, size)
}
var keySize int
algorithm := headers.getAlgorithm()
switch algorithm {
case ECDH_ES:
// ECDH-ES uses direct key agreement, no key unwrapping necessary.
return deriveKey(string(headers.getEncryption()), generator.keySize()), nil
case ECDH_ES_A128KW:
keySize = 16
case ECDH_ES_A192KW:
keySize = 24
case ECDH_ES_A256KW:
keySize = 32
default:
return nil, ErrUnsupportedAlgorithm
}
key := deriveKey(string(algorithm), keySize)
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
return josecipher.KeyUnwrap(block, recipient.encryptedKey)
}
func (ctx edDecrypterSigner) signPayload(payload []byte, alg SignatureAlgorithm) (Signature, error) {
if alg != EdDSA {
return Signature{}, ErrUnsupportedAlgorithm
}
sig, err := ctx.privateKey.Sign(RandReader, payload, crypto.Hash(0))
if err != nil {
return Signature{}, err
}
return Signature{
Signature: sig,
protected: &rawHeader{},
}, nil
}
func (ctx edEncrypterVerifier) verifyPayload(payload []byte, signature []byte, alg SignatureAlgorithm) error {
if alg != EdDSA {
return ErrUnsupportedAlgorithm
}
ok := ed25519.Verify(ctx.publicKey, payload, signature)
if !ok {
return errors.New("go-jose/go-jose: ed25519 signature failed to verify")
}
return nil
}
// Sign the given payload
func (ctx ecDecrypterSigner) signPayload(payload []byte, alg SignatureAlgorithm) (Signature, error) {
var expectedBitSize int
var hash crypto.Hash
switch alg {
case ES256:
expectedBitSize = 256
hash = crypto.SHA256
case ES384:
expectedBitSize = 384
hash = crypto.SHA384
case ES512:
expectedBitSize = 521
hash = crypto.SHA512
}
curveBits := ctx.privateKey.Curve.Params().BitSize
if expectedBitSize != curveBits {
return Signature{}, fmt.Errorf("go-jose/go-jose: expected %d bit key, got %d bits instead", expectedBitSize, curveBits)
}
hasher := hash.New()
// According to documentation, Write() on hash never fails
_, _ = hasher.Write(payload)
hashed := hasher.Sum(nil)
r, s, err := ecdsa.Sign(RandReader, ctx.privateKey, hashed)
if err != nil {
return Signature{}, err
}
keyBytes := curveBits / 8
if curveBits%8 > 0 {
keyBytes++
}
// We serialize the outputs (r and s) into big-endian byte arrays and pad
// them with zeros on the left to make sure the sizes work out. Both arrays
// must be keyBytes long, and the output must be 2*keyBytes long.
rBytes := r.Bytes()
rBytesPadded := make([]byte, keyBytes)
copy(rBytesPadded[keyBytes-len(rBytes):], rBytes)
sBytes := s.Bytes()
sBytesPadded := make([]byte, keyBytes)
copy(sBytesPadded[keyBytes-len(sBytes):], sBytes)
out := append(rBytesPadded, sBytesPadded...)
return Signature{
Signature: out,
protected: &rawHeader{},
}, nil
}
// Verify the given payload
func (ctx ecEncrypterVerifier) verifyPayload(payload []byte, signature []byte, alg SignatureAlgorithm) error {
var keySize int
var hash crypto.Hash
switch alg {
case ES256:
keySize = 32
hash = crypto.SHA256
case ES384:
keySize = 48
hash = crypto.SHA384
case ES512:
keySize = 66
hash = crypto.SHA512
default:
return ErrUnsupportedAlgorithm
}
if len(signature) != 2*keySize {
return fmt.Errorf("go-jose/go-jose: invalid signature size, have %d bytes, wanted %d", len(signature), 2*keySize)
}
hasher := hash.New()
// According to documentation, Write() on hash never fails
_, _ = hasher.Write(payload)
hashed := hasher.Sum(nil)
r := big.NewInt(0).SetBytes(signature[:keySize])
s := big.NewInt(0).SetBytes(signature[keySize:])
match := ecdsa.Verify(ctx.publicKey, hashed, r, s)
if !match {
return errors.New("go-jose/go-jose: ecdsa signature failed to verify")
}
return nil
}
+196
View File
@@ -0,0 +1,196 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package josecipher
import (
"bytes"
"crypto/cipher"
"crypto/hmac"
"crypto/sha256"
"crypto/sha512"
"crypto/subtle"
"encoding/binary"
"errors"
"hash"
)
const (
nonceBytes = 16
)
// NewCBCHMAC instantiates a new AEAD based on CBC+HMAC.
func NewCBCHMAC(key []byte, newBlockCipher func([]byte) (cipher.Block, error)) (cipher.AEAD, error) {
keySize := len(key) / 2
integrityKey := key[:keySize]
encryptionKey := key[keySize:]
blockCipher, err := newBlockCipher(encryptionKey)
if err != nil {
return nil, err
}
var hash func() hash.Hash
switch keySize {
case 16:
hash = sha256.New
case 24:
hash = sha512.New384
case 32:
hash = sha512.New
}
return &cbcAEAD{
hash: hash,
blockCipher: blockCipher,
authtagBytes: keySize,
integrityKey: integrityKey,
}, nil
}
// An AEAD based on CBC+HMAC
type cbcAEAD struct {
hash func() hash.Hash
authtagBytes int
integrityKey []byte
blockCipher cipher.Block
}
func (ctx *cbcAEAD) NonceSize() int {
return nonceBytes
}
func (ctx *cbcAEAD) Overhead() int {
// Maximum overhead is block size (for padding) plus auth tag length, where
// the length of the auth tag is equivalent to the key size.
return ctx.blockCipher.BlockSize() + ctx.authtagBytes
}
// Seal encrypts and authenticates the plaintext.
func (ctx *cbcAEAD) Seal(dst, nonce, plaintext, data []byte) []byte {
// Output buffer -- must take care not to mangle plaintext input.
ciphertext := make([]byte, uint64(len(plaintext))+uint64(ctx.Overhead()))[:len(plaintext)]
copy(ciphertext, plaintext)
ciphertext = padBuffer(ciphertext, ctx.blockCipher.BlockSize())
cbc := cipher.NewCBCEncrypter(ctx.blockCipher, nonce)
cbc.CryptBlocks(ciphertext, ciphertext)
authtag := ctx.computeAuthTag(data, nonce, ciphertext)
ret, out := resize(dst, uint64(len(dst))+uint64(len(ciphertext))+uint64(len(authtag)))
copy(out, ciphertext)
copy(out[len(ciphertext):], authtag)
return ret
}
// Open decrypts and authenticates the ciphertext.
func (ctx *cbcAEAD) Open(dst, nonce, ciphertext, data []byte) ([]byte, error) {
if len(ciphertext) < ctx.authtagBytes {
return nil, errors.New("go-jose/go-jose: invalid ciphertext (too short)")
}
offset := len(ciphertext) - ctx.authtagBytes
expectedTag := ctx.computeAuthTag(data, nonce, ciphertext[:offset])
match := subtle.ConstantTimeCompare(expectedTag, ciphertext[offset:])
if match != 1 {
return nil, errors.New("go-jose/go-jose: invalid ciphertext (auth tag mismatch)")
}
cbc := cipher.NewCBCDecrypter(ctx.blockCipher, nonce)
// Make copy of ciphertext buffer, don't want to modify in place
buffer := append([]byte{}, ciphertext[:offset]...)
if len(buffer)%ctx.blockCipher.BlockSize() > 0 {
return nil, errors.New("go-jose/go-jose: invalid ciphertext (invalid length)")
}
cbc.CryptBlocks(buffer, buffer)
// Remove padding
plaintext, err := unpadBuffer(buffer, ctx.blockCipher.BlockSize())
if err != nil {
return nil, err
}
ret, out := resize(dst, uint64(len(dst))+uint64(len(plaintext)))
copy(out, plaintext)
return ret, nil
}
// Compute an authentication tag
func (ctx *cbcAEAD) computeAuthTag(aad, nonce, ciphertext []byte) []byte {
buffer := make([]byte, uint64(len(aad))+uint64(len(nonce))+uint64(len(ciphertext))+8)
n := 0
n += copy(buffer, aad)
n += copy(buffer[n:], nonce)
n += copy(buffer[n:], ciphertext)
binary.BigEndian.PutUint64(buffer[n:], uint64(len(aad))*8)
// According to documentation, Write() on hash.Hash never fails.
hmac := hmac.New(ctx.hash, ctx.integrityKey)
_, _ = hmac.Write(buffer)
return hmac.Sum(nil)[:ctx.authtagBytes]
}
// resize ensures that the given slice has a capacity of at least n bytes.
// If the capacity of the slice is less than n, a new slice is allocated
// and the existing data will be copied.
func resize(in []byte, n uint64) (head, tail []byte) {
if uint64(cap(in)) >= n {
head = in[:n]
} else {
head = make([]byte, n)
copy(head, in)
}
tail = head[len(in):]
return
}
// Apply padding
func padBuffer(buffer []byte, blockSize int) []byte {
missing := blockSize - (len(buffer) % blockSize)
ret, out := resize(buffer, uint64(len(buffer))+uint64(missing))
padding := bytes.Repeat([]byte{byte(missing)}, missing)
copy(out, padding)
return ret
}
// Remove padding
func unpadBuffer(buffer []byte, blockSize int) ([]byte, error) {
if len(buffer)%blockSize != 0 {
return nil, errors.New("go-jose/go-jose: invalid padding")
}
last := buffer[len(buffer)-1]
count := int(last)
if count == 0 || count > blockSize || count > len(buffer) {
return nil, errors.New("go-jose/go-jose: invalid padding")
}
padding := bytes.Repeat([]byte{last}, count)
if !bytes.HasSuffix(buffer, padding) {
return nil, errors.New("go-jose/go-jose: invalid padding")
}
return buffer[:len(buffer)-count], nil
}
+75
View File
@@ -0,0 +1,75 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package josecipher
import (
"crypto"
"encoding/binary"
"hash"
"io"
)
type concatKDF struct {
z, info []byte
i uint32
cache []byte
hasher hash.Hash
}
// NewConcatKDF builds a KDF reader based on the given inputs.
func NewConcatKDF(hash crypto.Hash, z, algID, ptyUInfo, ptyVInfo, supPubInfo, supPrivInfo []byte) io.Reader {
buffer := make([]byte, uint64(len(algID))+uint64(len(ptyUInfo))+uint64(len(ptyVInfo))+uint64(len(supPubInfo))+uint64(len(supPrivInfo)))
n := 0
n += copy(buffer, algID)
n += copy(buffer[n:], ptyUInfo)
n += copy(buffer[n:], ptyVInfo)
n += copy(buffer[n:], supPubInfo)
copy(buffer[n:], supPrivInfo)
hasher := hash.New()
return &concatKDF{
z: z,
info: buffer,
hasher: hasher,
cache: []byte{},
i: 1,
}
}
func (ctx *concatKDF) Read(out []byte) (int, error) {
copied := copy(out, ctx.cache)
ctx.cache = ctx.cache[copied:]
for copied < len(out) {
ctx.hasher.Reset()
// Write on a hash.Hash never fails
_ = binary.Write(ctx.hasher, binary.BigEndian, ctx.i)
_, _ = ctx.hasher.Write(ctx.z)
_, _ = ctx.hasher.Write(ctx.info)
hash := ctx.hasher.Sum(nil)
chunkCopied := copy(out[copied:], hash)
copied += chunkCopied
ctx.cache = hash[chunkCopied:]
ctx.i++
}
return copied, nil
}
+86
View File
@@ -0,0 +1,86 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package josecipher
import (
"bytes"
"crypto"
"crypto/ecdsa"
"crypto/elliptic"
"encoding/binary"
)
// DeriveECDHES derives a shared encryption key using ECDH/ConcatKDF as described in JWE/JWA.
// It is an error to call this function with a private/public key that are not on the same
// curve. Callers must ensure that the keys are valid before calling this function. Output
// size may be at most 1<<16 bytes (64 KiB).
func DeriveECDHES(alg string, apuData, apvData []byte, priv *ecdsa.PrivateKey, pub *ecdsa.PublicKey, size int) []byte {
if size > 1<<16 {
panic("ECDH-ES output size too large, must be less than or equal to 1<<16")
}
// algId, partyUInfo, partyVInfo inputs must be prefixed with the length
algID := lengthPrefixed([]byte(alg))
ptyUInfo := lengthPrefixed(apuData)
ptyVInfo := lengthPrefixed(apvData)
// suppPubInfo is the encoded length of the output size in bits
supPubInfo := make([]byte, 4)
binary.BigEndian.PutUint32(supPubInfo, uint32(size)*8)
if !priv.PublicKey.Curve.IsOnCurve(pub.X, pub.Y) {
panic("public key not on same curve as private key")
}
z, _ := priv.Curve.ScalarMult(pub.X, pub.Y, priv.D.Bytes())
zBytes := z.Bytes()
// Note that calling z.Bytes() on a big.Int may strip leading zero bytes from
// the returned byte array. This can lead to a problem where zBytes will be
// shorter than expected which breaks the key derivation. Therefore we must pad
// to the full length of the expected coordinate here before calling the KDF.
octSize := dSize(priv.Curve)
if len(zBytes) != octSize {
zBytes = append(bytes.Repeat([]byte{0}, octSize-len(zBytes)), zBytes...)
}
reader := NewConcatKDF(crypto.SHA256, zBytes, algID, ptyUInfo, ptyVInfo, supPubInfo, []byte{})
key := make([]byte, size)
// Read on the KDF will never fail
_, _ = reader.Read(key)
return key
}
// dSize returns the size in octets for a coordinate on a elliptic curve.
func dSize(curve elliptic.Curve) int {
order := curve.Params().P
bitLen := order.BitLen()
size := bitLen / 8
if bitLen%8 != 0 {
size++
}
return size
}
func lengthPrefixed(data []byte) []byte {
out := make([]byte, len(data)+4)
binary.BigEndian.PutUint32(out, uint32(len(data)))
copy(out[4:], data)
return out
}
+109
View File
@@ -0,0 +1,109 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package josecipher
import (
"crypto/cipher"
"crypto/subtle"
"encoding/binary"
"errors"
)
var defaultIV = []byte{0xA6, 0xA6, 0xA6, 0xA6, 0xA6, 0xA6, 0xA6, 0xA6}
// KeyWrap implements NIST key wrapping; it wraps a content encryption key (cek) with the given block cipher.
func KeyWrap(block cipher.Block, cek []byte) ([]byte, error) {
if len(cek)%8 != 0 {
return nil, errors.New("go-jose/go-jose: key wrap input must be 8 byte blocks")
}
n := len(cek) / 8
r := make([][]byte, n)
for i := range r {
r[i] = make([]byte, 8)
copy(r[i], cek[i*8:])
}
buffer := make([]byte, 16)
tBytes := make([]byte, 8)
copy(buffer, defaultIV)
for t := 0; t < 6*n; t++ {
copy(buffer[8:], r[t%n])
block.Encrypt(buffer, buffer)
binary.BigEndian.PutUint64(tBytes, uint64(t+1))
for i := 0; i < 8; i++ {
buffer[i] ^= tBytes[i]
}
copy(r[t%n], buffer[8:])
}
out := make([]byte, (n+1)*8)
copy(out, buffer[:8])
for i := range r {
copy(out[(i+1)*8:], r[i])
}
return out, nil
}
// KeyUnwrap implements NIST key unwrapping; it unwraps a content encryption key (cek) with the given block cipher.
func KeyUnwrap(block cipher.Block, ciphertext []byte) ([]byte, error) {
if len(ciphertext)%8 != 0 {
return nil, errors.New("go-jose/go-jose: key wrap input must be 8 byte blocks")
}
n := (len(ciphertext) / 8) - 1
r := make([][]byte, n)
for i := range r {
r[i] = make([]byte, 8)
copy(r[i], ciphertext[(i+1)*8:])
}
buffer := make([]byte, 16)
tBytes := make([]byte, 8)
copy(buffer[:8], ciphertext[:8])
for t := 6*n - 1; t >= 0; t-- {
binary.BigEndian.PutUint64(tBytes, uint64(t+1))
for i := 0; i < 8; i++ {
buffer[i] ^= tBytes[i]
}
copy(buffer[8:], r[t%n])
block.Decrypt(buffer, buffer)
copy(r[t%n], buffer[8:])
}
if subtle.ConstantTimeCompare(buffer[:8], defaultIV) == 0 {
return nil, errors.New("go-jose/go-jose: failed to unwrap key")
}
out := make([]byte, n*8)
for i := range r {
copy(out[i*8:], r[i])
}
return out, nil
}
+544
View File
@@ -0,0 +1,544 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jose
import (
"crypto/ecdsa"
"crypto/rsa"
"errors"
"fmt"
"reflect"
"github.com/go-jose/go-jose/v3/json"
)
// Encrypter represents an encrypter which produces an encrypted JWE object.
type Encrypter interface {
Encrypt(plaintext []byte) (*JSONWebEncryption, error)
EncryptWithAuthData(plaintext []byte, aad []byte) (*JSONWebEncryption, error)
Options() EncrypterOptions
}
// A generic content cipher
type contentCipher interface {
keySize() int
encrypt(cek []byte, aad, plaintext []byte) (*aeadParts, error)
decrypt(cek []byte, aad []byte, parts *aeadParts) ([]byte, error)
}
// A key generator (for generating/getting a CEK)
type keyGenerator interface {
keySize() int
genKey() ([]byte, rawHeader, error)
}
// A generic key encrypter
type keyEncrypter interface {
encryptKey(cek []byte, alg KeyAlgorithm) (recipientInfo, error) // Encrypt a key
}
// A generic key decrypter
type keyDecrypter interface {
decryptKey(headers rawHeader, recipient *recipientInfo, generator keyGenerator) ([]byte, error) // Decrypt a key
}
// A generic encrypter based on the given key encrypter and content cipher.
type genericEncrypter struct {
contentAlg ContentEncryption
compressionAlg CompressionAlgorithm
cipher contentCipher
recipients []recipientKeyInfo
keyGenerator keyGenerator
extraHeaders map[HeaderKey]interface{}
}
type recipientKeyInfo struct {
keyID string
keyAlg KeyAlgorithm
keyEncrypter keyEncrypter
}
// EncrypterOptions represents options that can be set on new encrypters.
type EncrypterOptions struct {
Compression CompressionAlgorithm
// Optional map of additional keys to be inserted into the protected header
// of a JWS object. Some specifications which make use of JWS like to insert
// additional values here. All values must be JSON-serializable.
ExtraHeaders map[HeaderKey]interface{}
}
// WithHeader adds an arbitrary value to the ExtraHeaders map, initializing it
// if necessary. It returns itself and so can be used in a fluent style.
func (eo *EncrypterOptions) WithHeader(k HeaderKey, v interface{}) *EncrypterOptions {
if eo.ExtraHeaders == nil {
eo.ExtraHeaders = map[HeaderKey]interface{}{}
}
eo.ExtraHeaders[k] = v
return eo
}
// WithContentType adds a content type ("cty") header and returns the updated
// EncrypterOptions.
func (eo *EncrypterOptions) WithContentType(contentType ContentType) *EncrypterOptions {
return eo.WithHeader(HeaderContentType, contentType)
}
// WithType adds a type ("typ") header and returns the updated EncrypterOptions.
func (eo *EncrypterOptions) WithType(typ ContentType) *EncrypterOptions {
return eo.WithHeader(HeaderType, typ)
}
// Recipient represents an algorithm/key to encrypt messages to.
//
// PBES2Count and PBES2Salt correspond with the "p2c" and "p2s" headers used
// on the password-based encryption algorithms PBES2-HS256+A128KW,
// PBES2-HS384+A192KW, and PBES2-HS512+A256KW. If they are not provided a safe
// default of 100000 will be used for the count and a 128-bit random salt will
// be generated.
type Recipient struct {
Algorithm KeyAlgorithm
Key interface{}
KeyID string
PBES2Count int
PBES2Salt []byte
}
// NewEncrypter creates an appropriate encrypter based on the key type
func NewEncrypter(enc ContentEncryption, rcpt Recipient, opts *EncrypterOptions) (Encrypter, error) {
encrypter := &genericEncrypter{
contentAlg: enc,
recipients: []recipientKeyInfo{},
cipher: getContentCipher(enc),
}
if opts != nil {
encrypter.compressionAlg = opts.Compression
encrypter.extraHeaders = opts.ExtraHeaders
}
if encrypter.cipher == nil {
return nil, ErrUnsupportedAlgorithm
}
var keyID string
var rawKey interface{}
switch encryptionKey := rcpt.Key.(type) {
case JSONWebKey:
keyID, rawKey = encryptionKey.KeyID, encryptionKey.Key
case *JSONWebKey:
keyID, rawKey = encryptionKey.KeyID, encryptionKey.Key
case OpaqueKeyEncrypter:
keyID, rawKey = encryptionKey.KeyID(), encryptionKey
default:
rawKey = encryptionKey
}
switch rcpt.Algorithm {
case DIRECT:
// Direct encryption mode must be treated differently
if reflect.TypeOf(rawKey) != reflect.TypeOf([]byte{}) {
return nil, ErrUnsupportedKeyType
}
if encrypter.cipher.keySize() != len(rawKey.([]byte)) {
return nil, ErrInvalidKeySize
}
encrypter.keyGenerator = staticKeyGenerator{
key: rawKey.([]byte),
}
recipientInfo, _ := newSymmetricRecipient(rcpt.Algorithm, rawKey.([]byte))
recipientInfo.keyID = keyID
if rcpt.KeyID != "" {
recipientInfo.keyID = rcpt.KeyID
}
encrypter.recipients = []recipientKeyInfo{recipientInfo}
return encrypter, nil
case ECDH_ES:
// ECDH-ES (w/o key wrapping) is similar to DIRECT mode
typeOf := reflect.TypeOf(rawKey)
if typeOf != reflect.TypeOf(&ecdsa.PublicKey{}) {
return nil, ErrUnsupportedKeyType
}
encrypter.keyGenerator = ecKeyGenerator{
size: encrypter.cipher.keySize(),
algID: string(enc),
publicKey: rawKey.(*ecdsa.PublicKey),
}
recipientInfo, _ := newECDHRecipient(rcpt.Algorithm, rawKey.(*ecdsa.PublicKey))
recipientInfo.keyID = keyID
if rcpt.KeyID != "" {
recipientInfo.keyID = rcpt.KeyID
}
encrypter.recipients = []recipientKeyInfo{recipientInfo}
return encrypter, nil
default:
// Can just add a standard recipient
encrypter.keyGenerator = randomKeyGenerator{
size: encrypter.cipher.keySize(),
}
err := encrypter.addRecipient(rcpt)
return encrypter, err
}
}
// NewMultiEncrypter creates a multi-encrypter based on the given parameters
func NewMultiEncrypter(enc ContentEncryption, rcpts []Recipient, opts *EncrypterOptions) (Encrypter, error) {
cipher := getContentCipher(enc)
if cipher == nil {
return nil, ErrUnsupportedAlgorithm
}
if len(rcpts) == 0 {
return nil, fmt.Errorf("go-jose/go-jose: recipients is nil or empty")
}
encrypter := &genericEncrypter{
contentAlg: enc,
recipients: []recipientKeyInfo{},
cipher: cipher,
keyGenerator: randomKeyGenerator{
size: cipher.keySize(),
},
}
if opts != nil {
encrypter.compressionAlg = opts.Compression
encrypter.extraHeaders = opts.ExtraHeaders
}
for _, recipient := range rcpts {
err := encrypter.addRecipient(recipient)
if err != nil {
return nil, err
}
}
return encrypter, nil
}
func (ctx *genericEncrypter) addRecipient(recipient Recipient) (err error) {
var recipientInfo recipientKeyInfo
switch recipient.Algorithm {
case DIRECT, ECDH_ES:
return fmt.Errorf("go-jose/go-jose: key algorithm '%s' not supported in multi-recipient mode", recipient.Algorithm)
}
recipientInfo, err = makeJWERecipient(recipient.Algorithm, recipient.Key)
if recipient.KeyID != "" {
recipientInfo.keyID = recipient.KeyID
}
switch recipient.Algorithm {
case PBES2_HS256_A128KW, PBES2_HS384_A192KW, PBES2_HS512_A256KW:
if sr, ok := recipientInfo.keyEncrypter.(*symmetricKeyCipher); ok {
sr.p2c = recipient.PBES2Count
sr.p2s = recipient.PBES2Salt
}
}
if err == nil {
ctx.recipients = append(ctx.recipients, recipientInfo)
}
return err
}
func makeJWERecipient(alg KeyAlgorithm, encryptionKey interface{}) (recipientKeyInfo, error) {
switch encryptionKey := encryptionKey.(type) {
case *rsa.PublicKey:
return newRSARecipient(alg, encryptionKey)
case *ecdsa.PublicKey:
return newECDHRecipient(alg, encryptionKey)
case []byte:
return newSymmetricRecipient(alg, encryptionKey)
case string:
return newSymmetricRecipient(alg, []byte(encryptionKey))
case *JSONWebKey:
recipient, err := makeJWERecipient(alg, encryptionKey.Key)
recipient.keyID = encryptionKey.KeyID
return recipient, err
}
if encrypter, ok := encryptionKey.(OpaqueKeyEncrypter); ok {
return newOpaqueKeyEncrypter(alg, encrypter)
}
return recipientKeyInfo{}, ErrUnsupportedKeyType
}
// newDecrypter creates an appropriate decrypter based on the key type
func newDecrypter(decryptionKey interface{}) (keyDecrypter, error) {
switch decryptionKey := decryptionKey.(type) {
case *rsa.PrivateKey:
return &rsaDecrypterSigner{
privateKey: decryptionKey,
}, nil
case *ecdsa.PrivateKey:
return &ecDecrypterSigner{
privateKey: decryptionKey,
}, nil
case []byte:
return &symmetricKeyCipher{
key: decryptionKey,
}, nil
case string:
return &symmetricKeyCipher{
key: []byte(decryptionKey),
}, nil
case JSONWebKey:
return newDecrypter(decryptionKey.Key)
case *JSONWebKey:
return newDecrypter(decryptionKey.Key)
}
if okd, ok := decryptionKey.(OpaqueKeyDecrypter); ok {
return &opaqueKeyDecrypter{decrypter: okd}, nil
}
return nil, ErrUnsupportedKeyType
}
// Implementation of encrypt method producing a JWE object.
func (ctx *genericEncrypter) Encrypt(plaintext []byte) (*JSONWebEncryption, error) {
return ctx.EncryptWithAuthData(plaintext, nil)
}
// Implementation of encrypt method producing a JWE object.
func (ctx *genericEncrypter) EncryptWithAuthData(plaintext, aad []byte) (*JSONWebEncryption, error) {
obj := &JSONWebEncryption{}
obj.aad = aad
obj.protected = &rawHeader{}
err := obj.protected.set(headerEncryption, ctx.contentAlg)
if err != nil {
return nil, err
}
obj.recipients = make([]recipientInfo, len(ctx.recipients))
if len(ctx.recipients) == 0 {
return nil, fmt.Errorf("go-jose/go-jose: no recipients to encrypt to")
}
cek, headers, err := ctx.keyGenerator.genKey()
if err != nil {
return nil, err
}
obj.protected.merge(&headers)
for i, info := range ctx.recipients {
recipient, err := info.keyEncrypter.encryptKey(cek, info.keyAlg)
if err != nil {
return nil, err
}
err = recipient.header.set(headerAlgorithm, info.keyAlg)
if err != nil {
return nil, err
}
if info.keyID != "" {
err = recipient.header.set(headerKeyID, info.keyID)
if err != nil {
return nil, err
}
}
obj.recipients[i] = recipient
}
if len(ctx.recipients) == 1 {
// Move per-recipient headers into main protected header if there's
// only a single recipient.
obj.protected.merge(obj.recipients[0].header)
obj.recipients[0].header = nil
}
if ctx.compressionAlg != NONE {
plaintext, err = compress(ctx.compressionAlg, plaintext)
if err != nil {
return nil, err
}
err = obj.protected.set(headerCompression, ctx.compressionAlg)
if err != nil {
return nil, err
}
}
for k, v := range ctx.extraHeaders {
b, err := json.Marshal(v)
if err != nil {
return nil, err
}
(*obj.protected)[k] = makeRawMessage(b)
}
authData := obj.computeAuthData()
parts, err := ctx.cipher.encrypt(cek, authData, plaintext)
if err != nil {
return nil, err
}
obj.iv = parts.iv
obj.ciphertext = parts.ciphertext
obj.tag = parts.tag
return obj, nil
}
func (ctx *genericEncrypter) Options() EncrypterOptions {
return EncrypterOptions{
Compression: ctx.compressionAlg,
ExtraHeaders: ctx.extraHeaders,
}
}
// Decrypt and validate the object and return the plaintext. Note that this
// function does not support multi-recipient, if you desire multi-recipient
// decryption use DecryptMulti instead.
func (obj JSONWebEncryption) Decrypt(decryptionKey interface{}) ([]byte, error) {
headers := obj.mergedHeaders(nil)
if len(obj.recipients) > 1 {
return nil, errors.New("go-jose/go-jose: too many recipients in payload; expecting only one")
}
critical, err := headers.getCritical()
if err != nil {
return nil, fmt.Errorf("go-jose/go-jose: invalid crit header")
}
if len(critical) > 0 {
return nil, fmt.Errorf("go-jose/go-jose: unsupported crit header")
}
key := tryJWKS(decryptionKey, obj.Header)
decrypter, err := newDecrypter(key)
if err != nil {
return nil, err
}
cipher := getContentCipher(headers.getEncryption())
if cipher == nil {
return nil, fmt.Errorf("go-jose/go-jose: unsupported enc value '%s'", string(headers.getEncryption()))
}
generator := randomKeyGenerator{
size: cipher.keySize(),
}
parts := &aeadParts{
iv: obj.iv,
ciphertext: obj.ciphertext,
tag: obj.tag,
}
authData := obj.computeAuthData()
var plaintext []byte
recipient := obj.recipients[0]
recipientHeaders := obj.mergedHeaders(&recipient)
cek, err := decrypter.decryptKey(recipientHeaders, &recipient, generator)
if err == nil {
// Found a valid CEK -- let's try to decrypt.
plaintext, err = cipher.decrypt(cek, authData, parts)
}
if plaintext == nil {
return nil, ErrCryptoFailure
}
// The "zip" header parameter may only be present in the protected header.
if comp := obj.protected.getCompression(); comp != "" {
plaintext, err = decompress(comp, plaintext)
}
return plaintext, err
}
// DecryptMulti decrypts and validates the object and returns the plaintexts,
// with support for multiple recipients. It returns the index of the recipient
// for which the decryption was successful, the merged headers for that recipient,
// and the plaintext.
func (obj JSONWebEncryption) DecryptMulti(decryptionKey interface{}) (int, Header, []byte, error) {
globalHeaders := obj.mergedHeaders(nil)
critical, err := globalHeaders.getCritical()
if err != nil {
return -1, Header{}, nil, fmt.Errorf("go-jose/go-jose: invalid crit header")
}
if len(critical) > 0 {
return -1, Header{}, nil, fmt.Errorf("go-jose/go-jose: unsupported crit header")
}
key := tryJWKS(decryptionKey, obj.Header)
decrypter, err := newDecrypter(key)
if err != nil {
return -1, Header{}, nil, err
}
encryption := globalHeaders.getEncryption()
cipher := getContentCipher(encryption)
if cipher == nil {
return -1, Header{}, nil, fmt.Errorf("go-jose/go-jose: unsupported enc value '%s'", string(encryption))
}
generator := randomKeyGenerator{
size: cipher.keySize(),
}
parts := &aeadParts{
iv: obj.iv,
ciphertext: obj.ciphertext,
tag: obj.tag,
}
authData := obj.computeAuthData()
index := -1
var plaintext []byte
var headers rawHeader
for i, recipient := range obj.recipients {
recipientHeaders := obj.mergedHeaders(&recipient)
cek, err := decrypter.decryptKey(recipientHeaders, &recipient, generator)
if err == nil {
// Found a valid CEK -- let's try to decrypt.
plaintext, err = cipher.decrypt(cek, authData, parts)
if err == nil {
index = i
headers = recipientHeaders
break
}
}
}
if plaintext == nil {
return -1, Header{}, nil, ErrCryptoFailure
}
// The "zip" header parameter may only be present in the protected header.
if comp := obj.protected.getCompression(); comp != "" {
plaintext, _ = decompress(comp, plaintext)
}
sanitized, err := headers.sanitized()
if err != nil {
return -1, Header{}, nil, fmt.Errorf("go-jose/go-jose: failed to sanitize header: %v", err)
}
return index, sanitized, plaintext, err
}
+27
View File
@@ -0,0 +1,27 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*
Package jose aims to provide an implementation of the Javascript Object Signing
and Encryption set of standards. It implements encryption and signing based on
the JSON Web Encryption and JSON Web Signature standards, with optional JSON Web
Token support available in a sub-package. The library supports both the compact
and JWS/JWE JSON Serialization formats, and has optional support for multiple
recipients.
*/
package jose
+191
View File
@@ -0,0 +1,191 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jose
import (
"bytes"
"compress/flate"
"encoding/base64"
"encoding/binary"
"io"
"math/big"
"strings"
"unicode"
"github.com/go-jose/go-jose/v3/json"
)
// Helper function to serialize known-good objects.
// Precondition: value is not a nil pointer.
func mustSerializeJSON(value interface{}) []byte {
out, err := json.Marshal(value)
if err != nil {
panic(err)
}
// We never want to serialize the top-level value "null," since it's not a
// valid JOSE message. But if a caller passes in a nil pointer to this method,
// MarshalJSON will happily serialize it as the top-level value "null". If
// that value is then embedded in another operation, for instance by being
// base64-encoded and fed as input to a signing algorithm
// (https://github.com/go-jose/go-jose/issues/22), the result will be
// incorrect. Because this method is intended for known-good objects, and a nil
// pointer is not a known-good object, we are free to panic in this case.
// Note: It's not possible to directly check whether the data pointed at by an
// interface is a nil pointer, so we do this hacky workaround.
// https://groups.google.com/forum/#!topic/golang-nuts/wnH302gBa4I
if string(out) == "null" {
panic("Tried to serialize a nil pointer.")
}
return out
}
// Strip all newlines and whitespace
func stripWhitespace(data string) string {
buf := strings.Builder{}
buf.Grow(len(data))
for _, r := range data {
if !unicode.IsSpace(r) {
buf.WriteRune(r)
}
}
return buf.String()
}
// Perform compression based on algorithm
func compress(algorithm CompressionAlgorithm, input []byte) ([]byte, error) {
switch algorithm {
case DEFLATE:
return deflate(input)
default:
return nil, ErrUnsupportedAlgorithm
}
}
// Perform decompression based on algorithm
func decompress(algorithm CompressionAlgorithm, input []byte) ([]byte, error) {
switch algorithm {
case DEFLATE:
return inflate(input)
default:
return nil, ErrUnsupportedAlgorithm
}
}
// Compress with DEFLATE
func deflate(input []byte) ([]byte, error) {
output := new(bytes.Buffer)
// Writing to byte buffer, err is always nil
writer, _ := flate.NewWriter(output, 1)
_, _ = io.Copy(writer, bytes.NewBuffer(input))
err := writer.Close()
return output.Bytes(), err
}
// Decompress with DEFLATE
func inflate(input []byte) ([]byte, error) {
output := new(bytes.Buffer)
reader := flate.NewReader(bytes.NewBuffer(input))
_, err := io.Copy(output, reader)
if err != nil {
return nil, err
}
err = reader.Close()
return output.Bytes(), err
}
// byteBuffer represents a slice of bytes that can be serialized to url-safe base64.
type byteBuffer struct {
data []byte
}
func newBuffer(data []byte) *byteBuffer {
if data == nil {
return nil
}
return &byteBuffer{
data: data,
}
}
func newFixedSizeBuffer(data []byte, length int) *byteBuffer {
if len(data) > length {
panic("go-jose/go-jose: invalid call to newFixedSizeBuffer (len(data) > length)")
}
pad := make([]byte, length-len(data))
return newBuffer(append(pad, data...))
}
func newBufferFromInt(num uint64) *byteBuffer {
data := make([]byte, 8)
binary.BigEndian.PutUint64(data, num)
return newBuffer(bytes.TrimLeft(data, "\x00"))
}
func (b *byteBuffer) MarshalJSON() ([]byte, error) {
return json.Marshal(b.base64())
}
func (b *byteBuffer) UnmarshalJSON(data []byte) error {
var encoded string
err := json.Unmarshal(data, &encoded)
if err != nil {
return err
}
if encoded == "" {
return nil
}
decoded, err := base64URLDecode(encoded)
if err != nil {
return err
}
*b = *newBuffer(decoded)
return nil
}
func (b *byteBuffer) base64() string {
return base64.RawURLEncoding.EncodeToString(b.data)
}
func (b *byteBuffer) bytes() []byte {
// Handling nil here allows us to transparently handle nil slices when serializing.
if b == nil {
return nil
}
return b.data
}
func (b byteBuffer) bigInt() *big.Int {
return new(big.Int).SetBytes(b.data)
}
func (b byteBuffer) toInt() int {
return int(b.bigInt().Int64())
}
// base64URLDecode is implemented as defined in https://www.rfc-editor.org/rfc/rfc7515.html#appendix-C
func base64URLDecode(value string) ([]byte, error) {
value = strings.TrimRight(value, "=")
return base64.RawURLEncoding.DecodeString(value)
}
+27
View File
@@ -0,0 +1,27 @@
Copyright (c) 2012 The Go Authors. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following disclaimer
in the documentation and/or other materials provided with the
distribution.
* Neither the name of Google Inc. nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+13
View File
@@ -0,0 +1,13 @@
# Safe JSON
This repository contains a fork of the `encoding/json` package from Go 1.6.
The following changes were made:
* Object deserialization uses case-sensitive member name matching instead of
[case-insensitive matching](https://www.ietf.org/mail-archive/web/json/current/msg03763.html).
This is to avoid differences in the interpretation of JOSE messages between
go-jose and libraries written in other languages.
* When deserializing a JSON object, we check for duplicate keys and reject the
input whenever we detect a duplicate. Rather than trying to work with malformed
data, we prefer to reject it right away.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+141
View File
@@ -0,0 +1,141 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package json
import "bytes"
// Compact appends to dst the JSON-encoded src with
// insignificant space characters elided.
func Compact(dst *bytes.Buffer, src []byte) error {
return compact(dst, src, false)
}
func compact(dst *bytes.Buffer, src []byte, escape bool) error {
origLen := dst.Len()
var scan scanner
scan.reset()
start := 0
for i, c := range src {
if escape && (c == '<' || c == '>' || c == '&') {
if start < i {
dst.Write(src[start:i])
}
dst.WriteString(`\u00`)
dst.WriteByte(hex[c>>4])
dst.WriteByte(hex[c&0xF])
start = i + 1
}
// Convert U+2028 and U+2029 (E2 80 A8 and E2 80 A9).
if c == 0xE2 && i+2 < len(src) && src[i+1] == 0x80 && src[i+2]&^1 == 0xA8 {
if start < i {
dst.Write(src[start:i])
}
dst.WriteString(`\u202`)
dst.WriteByte(hex[src[i+2]&0xF])
start = i + 3
}
v := scan.step(&scan, c)
if v >= scanSkipSpace {
if v == scanError {
break
}
if start < i {
dst.Write(src[start:i])
}
start = i + 1
}
}
if scan.eof() == scanError {
dst.Truncate(origLen)
return scan.err
}
if start < len(src) {
dst.Write(src[start:])
}
return nil
}
func newline(dst *bytes.Buffer, prefix, indent string, depth int) {
dst.WriteByte('\n')
dst.WriteString(prefix)
for i := 0; i < depth; i++ {
dst.WriteString(indent)
}
}
// Indent appends to dst an indented form of the JSON-encoded src.
// Each element in a JSON object or array begins on a new,
// indented line beginning with prefix followed by one or more
// copies of indent according to the indentation nesting.
// The data appended to dst does not begin with the prefix nor
// any indentation, to make it easier to embed inside other formatted JSON data.
// Although leading space characters (space, tab, carriage return, newline)
// at the beginning of src are dropped, trailing space characters
// at the end of src are preserved and copied to dst.
// For example, if src has no trailing spaces, neither will dst;
// if src ends in a trailing newline, so will dst.
func Indent(dst *bytes.Buffer, src []byte, prefix, indent string) error {
origLen := dst.Len()
var scan scanner
scan.reset()
needIndent := false
depth := 0
for _, c := range src {
scan.bytes++
v := scan.step(&scan, c)
if v == scanSkipSpace {
continue
}
if v == scanError {
break
}
if needIndent && v != scanEndObject && v != scanEndArray {
needIndent = false
depth++
newline(dst, prefix, indent, depth)
}
// Emit semantically uninteresting bytes
// (in particular, punctuation in strings) unmodified.
if v == scanContinue {
dst.WriteByte(c)
continue
}
// Add spacing around real punctuation.
switch c {
case '{', '[':
// delay indent so that empty object and array are formatted as {} and [].
needIndent = true
dst.WriteByte(c)
case ',':
dst.WriteByte(c)
newline(dst, prefix, indent, depth)
case ':':
dst.WriteByte(c)
dst.WriteByte(' ')
case '}', ']':
if needIndent {
// suppress indent in empty object/array
needIndent = false
} else {
depth--
newline(dst, prefix, indent, depth)
}
dst.WriteByte(c)
default:
dst.WriteByte(c)
}
}
if scan.eof() == scanError {
dst.Truncate(origLen)
return scan.err
}
return nil
}
+623
View File
@@ -0,0 +1,623 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package json
// JSON value parser state machine.
// Just about at the limit of what is reasonable to write by hand.
// Some parts are a bit tedious, but overall it nicely factors out the
// otherwise common code from the multiple scanning functions
// in this package (Compact, Indent, checkValid, nextValue, etc).
//
// This file starts with two simple examples using the scanner
// before diving into the scanner itself.
import "strconv"
// checkValid verifies that data is valid JSON-encoded data.
// scan is passed in for use by checkValid to avoid an allocation.
func checkValid(data []byte, scan *scanner) error {
scan.reset()
for _, c := range data {
scan.bytes++
if scan.step(scan, c) == scanError {
return scan.err
}
}
if scan.eof() == scanError {
return scan.err
}
return nil
}
// nextValue splits data after the next whole JSON value,
// returning that value and the bytes that follow it as separate slices.
// scan is passed in for use by nextValue to avoid an allocation.
func nextValue(data []byte, scan *scanner) (value, rest []byte, err error) {
scan.reset()
for i, c := range data {
v := scan.step(scan, c)
if v >= scanEndObject {
switch v {
// probe the scanner with a space to determine whether we will
// get scanEnd on the next character. Otherwise, if the next character
// is not a space, scanEndTop allocates a needless error.
case scanEndObject, scanEndArray:
if scan.step(scan, ' ') == scanEnd {
return data[:i+1], data[i+1:], nil
}
case scanError:
return nil, nil, scan.err
case scanEnd:
return data[:i], data[i:], nil
}
}
}
if scan.eof() == scanError {
return nil, nil, scan.err
}
return data, nil, nil
}
// A SyntaxError is a description of a JSON syntax error.
type SyntaxError struct {
msg string // description of error
Offset int64 // error occurred after reading Offset bytes
}
func (e *SyntaxError) Error() string { return e.msg }
// A scanner is a JSON scanning state machine.
// Callers call scan.reset() and then pass bytes in one at a time
// by calling scan.step(&scan, c) for each byte.
// The return value, referred to as an opcode, tells the
// caller about significant parsing events like beginning
// and ending literals, objects, and arrays, so that the
// caller can follow along if it wishes.
// The return value scanEnd indicates that a single top-level
// JSON value has been completed, *before* the byte that
// just got passed in. (The indication must be delayed in order
// to recognize the end of numbers: is 123 a whole value or
// the beginning of 12345e+6?).
type scanner struct {
// The step is a func to be called to execute the next transition.
// Also tried using an integer constant and a single func
// with a switch, but using the func directly was 10% faster
// on a 64-bit Mac Mini, and it's nicer to read.
step func(*scanner, byte) int
// Reached end of top-level value.
endTop bool
// Stack of what we're in the middle of - array values, object keys, object values.
parseState []int
// Error that happened, if any.
err error
// 1-byte redo (see undo method)
redo bool
redoCode int
redoState func(*scanner, byte) int
// total bytes consumed, updated by decoder.Decode
bytes int64
}
// These values are returned by the state transition functions
// assigned to scanner.state and the method scanner.eof.
// They give details about the current state of the scan that
// callers might be interested to know about.
// It is okay to ignore the return value of any particular
// call to scanner.state: if one call returns scanError,
// every subsequent call will return scanError too.
const (
// Continue.
scanContinue = iota // uninteresting byte
scanBeginLiteral // end implied by next result != scanContinue
scanBeginObject // begin object
scanObjectKey // just finished object key (string)
scanObjectValue // just finished non-last object value
scanEndObject // end object (implies scanObjectValue if possible)
scanBeginArray // begin array
scanArrayValue // just finished array value
scanEndArray // end array (implies scanArrayValue if possible)
scanSkipSpace // space byte; can skip; known to be last "continue" result
// Stop.
scanEnd // top-level value ended *before* this byte; known to be first "stop" result
scanError // hit an error, scanner.err.
)
// These values are stored in the parseState stack.
// They give the current state of a composite value
// being scanned. If the parser is inside a nested value
// the parseState describes the nested state, outermost at entry 0.
const (
parseObjectKey = iota // parsing object key (before colon)
parseObjectValue // parsing object value (after colon)
parseArrayValue // parsing array value
)
// reset prepares the scanner for use.
// It must be called before calling s.step.
func (s *scanner) reset() {
s.step = stateBeginValue
s.parseState = s.parseState[0:0]
s.err = nil
s.redo = false
s.endTop = false
}
// eof tells the scanner that the end of input has been reached.
// It returns a scan status just as s.step does.
func (s *scanner) eof() int {
if s.err != nil {
return scanError
}
if s.endTop {
return scanEnd
}
s.step(s, ' ')
if s.endTop {
return scanEnd
}
if s.err == nil {
s.err = &SyntaxError{"unexpected end of JSON input", s.bytes}
}
return scanError
}
// pushParseState pushes a new parse state p onto the parse stack.
func (s *scanner) pushParseState(p int) {
s.parseState = append(s.parseState, p)
}
// popParseState pops a parse state (already obtained) off the stack
// and updates s.step accordingly.
func (s *scanner) popParseState() {
n := len(s.parseState) - 1
s.parseState = s.parseState[0:n]
s.redo = false
if n == 0 {
s.step = stateEndTop
s.endTop = true
} else {
s.step = stateEndValue
}
}
func isSpace(c byte) bool {
return c == ' ' || c == '\t' || c == '\r' || c == '\n'
}
// stateBeginValueOrEmpty is the state after reading `[`.
func stateBeginValueOrEmpty(s *scanner, c byte) int {
if c <= ' ' && isSpace(c) {
return scanSkipSpace
}
if c == ']' {
return stateEndValue(s, c)
}
return stateBeginValue(s, c)
}
// stateBeginValue is the state at the beginning of the input.
func stateBeginValue(s *scanner, c byte) int {
if c <= ' ' && isSpace(c) {
return scanSkipSpace
}
switch c {
case '{':
s.step = stateBeginStringOrEmpty
s.pushParseState(parseObjectKey)
return scanBeginObject
case '[':
s.step = stateBeginValueOrEmpty
s.pushParseState(parseArrayValue)
return scanBeginArray
case '"':
s.step = stateInString
return scanBeginLiteral
case '-':
s.step = stateNeg
return scanBeginLiteral
case '0': // beginning of 0.123
s.step = state0
return scanBeginLiteral
case 't': // beginning of true
s.step = stateT
return scanBeginLiteral
case 'f': // beginning of false
s.step = stateF
return scanBeginLiteral
case 'n': // beginning of null
s.step = stateN
return scanBeginLiteral
}
if '1' <= c && c <= '9' { // beginning of 1234.5
s.step = state1
return scanBeginLiteral
}
return s.error(c, "looking for beginning of value")
}
// stateBeginStringOrEmpty is the state after reading `{`.
func stateBeginStringOrEmpty(s *scanner, c byte) int {
if c <= ' ' && isSpace(c) {
return scanSkipSpace
}
if c == '}' {
n := len(s.parseState)
s.parseState[n-1] = parseObjectValue
return stateEndValue(s, c)
}
return stateBeginString(s, c)
}
// stateBeginString is the state after reading `{"key": value,`.
func stateBeginString(s *scanner, c byte) int {
if c <= ' ' && isSpace(c) {
return scanSkipSpace
}
if c == '"' {
s.step = stateInString
return scanBeginLiteral
}
return s.error(c, "looking for beginning of object key string")
}
// stateEndValue is the state after completing a value,
// such as after reading `{}` or `true` or `["x"`.
func stateEndValue(s *scanner, c byte) int {
n := len(s.parseState)
if n == 0 {
// Completed top-level before the current byte.
s.step = stateEndTop
s.endTop = true
return stateEndTop(s, c)
}
if c <= ' ' && isSpace(c) {
s.step = stateEndValue
return scanSkipSpace
}
ps := s.parseState[n-1]
switch ps {
case parseObjectKey:
if c == ':' {
s.parseState[n-1] = parseObjectValue
s.step = stateBeginValue
return scanObjectKey
}
return s.error(c, "after object key")
case parseObjectValue:
if c == ',' {
s.parseState[n-1] = parseObjectKey
s.step = stateBeginString
return scanObjectValue
}
if c == '}' {
s.popParseState()
return scanEndObject
}
return s.error(c, "after object key:value pair")
case parseArrayValue:
if c == ',' {
s.step = stateBeginValue
return scanArrayValue
}
if c == ']' {
s.popParseState()
return scanEndArray
}
return s.error(c, "after array element")
}
return s.error(c, "")
}
// stateEndTop is the state after finishing the top-level value,
// such as after reading `{}` or `[1,2,3]`.
// Only space characters should be seen now.
func stateEndTop(s *scanner, c byte) int {
if c != ' ' && c != '\t' && c != '\r' && c != '\n' {
// Complain about non-space byte on next call.
s.error(c, "after top-level value")
}
return scanEnd
}
// stateInString is the state after reading `"`.
func stateInString(s *scanner, c byte) int {
if c == '"' {
s.step = stateEndValue
return scanContinue
}
if c == '\\' {
s.step = stateInStringEsc
return scanContinue
}
if c < 0x20 {
return s.error(c, "in string literal")
}
return scanContinue
}
// stateInStringEsc is the state after reading `"\` during a quoted string.
func stateInStringEsc(s *scanner, c byte) int {
switch c {
case 'b', 'f', 'n', 'r', 't', '\\', '/', '"':
s.step = stateInString
return scanContinue
case 'u':
s.step = stateInStringEscU
return scanContinue
}
return s.error(c, "in string escape code")
}
// stateInStringEscU is the state after reading `"\u` during a quoted string.
func stateInStringEscU(s *scanner, c byte) int {
if '0' <= c && c <= '9' || 'a' <= c && c <= 'f' || 'A' <= c && c <= 'F' {
s.step = stateInStringEscU1
return scanContinue
}
// numbers
return s.error(c, "in \\u hexadecimal character escape")
}
// stateInStringEscU1 is the state after reading `"\u1` during a quoted string.
func stateInStringEscU1(s *scanner, c byte) int {
if '0' <= c && c <= '9' || 'a' <= c && c <= 'f' || 'A' <= c && c <= 'F' {
s.step = stateInStringEscU12
return scanContinue
}
// numbers
return s.error(c, "in \\u hexadecimal character escape")
}
// stateInStringEscU12 is the state after reading `"\u12` during a quoted string.
func stateInStringEscU12(s *scanner, c byte) int {
if '0' <= c && c <= '9' || 'a' <= c && c <= 'f' || 'A' <= c && c <= 'F' {
s.step = stateInStringEscU123
return scanContinue
}
// numbers
return s.error(c, "in \\u hexadecimal character escape")
}
// stateInStringEscU123 is the state after reading `"\u123` during a quoted string.
func stateInStringEscU123(s *scanner, c byte) int {
if '0' <= c && c <= '9' || 'a' <= c && c <= 'f' || 'A' <= c && c <= 'F' {
s.step = stateInString
return scanContinue
}
// numbers
return s.error(c, "in \\u hexadecimal character escape")
}
// stateNeg is the state after reading `-` during a number.
func stateNeg(s *scanner, c byte) int {
if c == '0' {
s.step = state0
return scanContinue
}
if '1' <= c && c <= '9' {
s.step = state1
return scanContinue
}
return s.error(c, "in numeric literal")
}
// state1 is the state after reading a non-zero integer during a number,
// such as after reading `1` or `100` but not `0`.
func state1(s *scanner, c byte) int {
if '0' <= c && c <= '9' {
s.step = state1
return scanContinue
}
return state0(s, c)
}
// state0 is the state after reading `0` during a number.
func state0(s *scanner, c byte) int {
if c == '.' {
s.step = stateDot
return scanContinue
}
if c == 'e' || c == 'E' {
s.step = stateE
return scanContinue
}
return stateEndValue(s, c)
}
// stateDot is the state after reading the integer and decimal point in a number,
// such as after reading `1.`.
func stateDot(s *scanner, c byte) int {
if '0' <= c && c <= '9' {
s.step = stateDot0
return scanContinue
}
return s.error(c, "after decimal point in numeric literal")
}
// stateDot0 is the state after reading the integer, decimal point, and subsequent
// digits of a number, such as after reading `3.14`.
func stateDot0(s *scanner, c byte) int {
if '0' <= c && c <= '9' {
return scanContinue
}
if c == 'e' || c == 'E' {
s.step = stateE
return scanContinue
}
return stateEndValue(s, c)
}
// stateE is the state after reading the mantissa and e in a number,
// such as after reading `314e` or `0.314e`.
func stateE(s *scanner, c byte) int {
if c == '+' || c == '-' {
s.step = stateESign
return scanContinue
}
return stateESign(s, c)
}
// stateESign is the state after reading the mantissa, e, and sign in a number,
// such as after reading `314e-` or `0.314e+`.
func stateESign(s *scanner, c byte) int {
if '0' <= c && c <= '9' {
s.step = stateE0
return scanContinue
}
return s.error(c, "in exponent of numeric literal")
}
// stateE0 is the state after reading the mantissa, e, optional sign,
// and at least one digit of the exponent in a number,
// such as after reading `314e-2` or `0.314e+1` or `3.14e0`.
func stateE0(s *scanner, c byte) int {
if '0' <= c && c <= '9' {
return scanContinue
}
return stateEndValue(s, c)
}
// stateT is the state after reading `t`.
func stateT(s *scanner, c byte) int {
if c == 'r' {
s.step = stateTr
return scanContinue
}
return s.error(c, "in literal true (expecting 'r')")
}
// stateTr is the state after reading `tr`.
func stateTr(s *scanner, c byte) int {
if c == 'u' {
s.step = stateTru
return scanContinue
}
return s.error(c, "in literal true (expecting 'u')")
}
// stateTru is the state after reading `tru`.
func stateTru(s *scanner, c byte) int {
if c == 'e' {
s.step = stateEndValue
return scanContinue
}
return s.error(c, "in literal true (expecting 'e')")
}
// stateF is the state after reading `f`.
func stateF(s *scanner, c byte) int {
if c == 'a' {
s.step = stateFa
return scanContinue
}
return s.error(c, "in literal false (expecting 'a')")
}
// stateFa is the state after reading `fa`.
func stateFa(s *scanner, c byte) int {
if c == 'l' {
s.step = stateFal
return scanContinue
}
return s.error(c, "in literal false (expecting 'l')")
}
// stateFal is the state after reading `fal`.
func stateFal(s *scanner, c byte) int {
if c == 's' {
s.step = stateFals
return scanContinue
}
return s.error(c, "in literal false (expecting 's')")
}
// stateFals is the state after reading `fals`.
func stateFals(s *scanner, c byte) int {
if c == 'e' {
s.step = stateEndValue
return scanContinue
}
return s.error(c, "in literal false (expecting 'e')")
}
// stateN is the state after reading `n`.
func stateN(s *scanner, c byte) int {
if c == 'u' {
s.step = stateNu
return scanContinue
}
return s.error(c, "in literal null (expecting 'u')")
}
// stateNu is the state after reading `nu`.
func stateNu(s *scanner, c byte) int {
if c == 'l' {
s.step = stateNul
return scanContinue
}
return s.error(c, "in literal null (expecting 'l')")
}
// stateNul is the state after reading `nul`.
func stateNul(s *scanner, c byte) int {
if c == 'l' {
s.step = stateEndValue
return scanContinue
}
return s.error(c, "in literal null (expecting 'l')")
}
// stateError is the state after reaching a syntax error,
// such as after reading `[1}` or `5.1.2`.
func stateError(s *scanner, c byte) int {
return scanError
}
// error records an error and switches to the error state.
func (s *scanner) error(c byte, context string) int {
s.step = stateError
s.err = &SyntaxError{"invalid character " + quoteChar(c) + " " + context, s.bytes}
return scanError
}
// quoteChar formats c as a quoted character literal
func quoteChar(c byte) string {
// special cases - different from quoted strings
if c == '\'' {
return `'\''`
}
if c == '"' {
return `'"'`
}
// use quoted string with different quotation marks
s := strconv.Quote(string(c))
return "'" + s[1:len(s)-1] + "'"
}
// undo causes the scanner to return scanCode from the next state transition.
// This gives callers a simple 1-byte undo mechanism.
func (s *scanner) undo(scanCode int) {
if s.redo {
panic("json: invalid use of scanner")
}
s.redoCode = scanCode
s.redoState = s.step
s.step = stateRedo
s.redo = true
}
// stateRedo helps implement the scanner's 1-byte undo.
func stateRedo(s *scanner, c byte) int {
s.redo = false
s.step = s.redoState
return s.redoCode
}
+485
View File
@@ -0,0 +1,485 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package json
import (
"bytes"
"errors"
"io"
)
// A Decoder reads and decodes JSON objects from an input stream.
type Decoder struct {
r io.Reader
buf []byte
d decodeState
scanp int // start of unread data in buf
scan scanner
err error
tokenState int
tokenStack []int
}
// NewDecoder returns a new decoder that reads from r.
//
// The decoder introduces its own buffering and may
// read data from r beyond the JSON values requested.
func NewDecoder(r io.Reader) *Decoder {
return &Decoder{r: r}
}
// Deprecated: Use `SetNumberType` instead
// UseNumber causes the Decoder to unmarshal a number into an interface{} as a
// Number instead of as a float64.
func (dec *Decoder) UseNumber() { dec.d.numberType = UnmarshalJSONNumber }
// SetNumberType causes the Decoder to unmarshal a number into an interface{} as a
// Number, float64 or int64 depending on `t` enum value.
func (dec *Decoder) SetNumberType(t NumberUnmarshalType) { dec.d.numberType = t }
// Decode reads the next JSON-encoded value from its
// input and stores it in the value pointed to by v.
//
// See the documentation for Unmarshal for details about
// the conversion of JSON into a Go value.
func (dec *Decoder) Decode(v interface{}) error {
if dec.err != nil {
return dec.err
}
if err := dec.tokenPrepareForDecode(); err != nil {
return err
}
if !dec.tokenValueAllowed() {
return &SyntaxError{msg: "not at beginning of value"}
}
// Read whole value into buffer.
n, err := dec.readValue()
if err != nil {
return err
}
dec.d.init(dec.buf[dec.scanp : dec.scanp+n])
dec.scanp += n
// Don't save err from unmarshal into dec.err:
// the connection is still usable since we read a complete JSON
// object from it before the error happened.
err = dec.d.unmarshal(v)
// fixup token streaming state
dec.tokenValueEnd()
return err
}
// Buffered returns a reader of the data remaining in the Decoder's
// buffer. The reader is valid until the next call to Decode.
func (dec *Decoder) Buffered() io.Reader {
return bytes.NewReader(dec.buf[dec.scanp:])
}
// readValue reads a JSON value into dec.buf.
// It returns the length of the encoding.
func (dec *Decoder) readValue() (int, error) {
dec.scan.reset()
scanp := dec.scanp
var err error
Input:
for {
// Look in the buffer for a new value.
for i, c := range dec.buf[scanp:] {
dec.scan.bytes++
v := dec.scan.step(&dec.scan, c)
if v == scanEnd {
scanp += i
break Input
}
// scanEnd is delayed one byte.
// We might block trying to get that byte from src,
// so instead invent a space byte.
if (v == scanEndObject || v == scanEndArray) && dec.scan.step(&dec.scan, ' ') == scanEnd {
scanp += i + 1
break Input
}
if v == scanError {
dec.err = dec.scan.err
return 0, dec.scan.err
}
}
scanp = len(dec.buf)
// Did the last read have an error?
// Delayed until now to allow buffer scan.
if err != nil {
if err == io.EOF {
if dec.scan.step(&dec.scan, ' ') == scanEnd {
break Input
}
if nonSpace(dec.buf) {
err = io.ErrUnexpectedEOF
}
}
dec.err = err
return 0, err
}
n := scanp - dec.scanp
err = dec.refill()
scanp = dec.scanp + n
}
return scanp - dec.scanp, nil
}
func (dec *Decoder) refill() error {
// Make room to read more into the buffer.
// First slide down data already consumed.
if dec.scanp > 0 {
n := copy(dec.buf, dec.buf[dec.scanp:])
dec.buf = dec.buf[:n]
dec.scanp = 0
}
// Grow buffer if not large enough.
const minRead = 512
if cap(dec.buf)-len(dec.buf) < minRead {
newBuf := make([]byte, len(dec.buf), 2*cap(dec.buf)+minRead)
copy(newBuf, dec.buf)
dec.buf = newBuf
}
// Read. Delay error for next iteration (after scan).
n, err := dec.r.Read(dec.buf[len(dec.buf):cap(dec.buf)])
dec.buf = dec.buf[0 : len(dec.buf)+n]
return err
}
func nonSpace(b []byte) bool {
for _, c := range b {
if !isSpace(c) {
return true
}
}
return false
}
// An Encoder writes JSON objects to an output stream.
type Encoder struct {
w io.Writer
err error
}
// NewEncoder returns a new encoder that writes to w.
func NewEncoder(w io.Writer) *Encoder {
return &Encoder{w: w}
}
// Encode writes the JSON encoding of v to the stream,
// followed by a newline character.
//
// See the documentation for Marshal for details about the
// conversion of Go values to JSON.
func (enc *Encoder) Encode(v interface{}) error {
if enc.err != nil {
return enc.err
}
e := newEncodeState()
err := e.marshal(v)
if err != nil {
return err
}
// Terminate each value with a newline.
// This makes the output look a little nicer
// when debugging, and some kind of space
// is required if the encoded value was a number,
// so that the reader knows there aren't more
// digits coming.
e.WriteByte('\n')
if _, err = enc.w.Write(e.Bytes()); err != nil {
enc.err = err
}
encodeStatePool.Put(e)
return err
}
// RawMessage is a raw encoded JSON object.
// It implements Marshaler and Unmarshaler and can
// be used to delay JSON decoding or precompute a JSON encoding.
type RawMessage []byte
// MarshalJSON returns *m as the JSON encoding of m.
func (m *RawMessage) MarshalJSON() ([]byte, error) {
return *m, nil
}
// UnmarshalJSON sets *m to a copy of data.
func (m *RawMessage) UnmarshalJSON(data []byte) error {
if m == nil {
return errors.New("json.RawMessage: UnmarshalJSON on nil pointer")
}
*m = append((*m)[0:0], data...)
return nil
}
var _ Marshaler = (*RawMessage)(nil)
var _ Unmarshaler = (*RawMessage)(nil)
// A Token holds a value of one of these types:
//
// Delim, for the four JSON delimiters [ ] { }
// bool, for JSON booleans
// float64, for JSON numbers
// Number, for JSON numbers
// string, for JSON string literals
// nil, for JSON null
//
type Token interface{}
const (
tokenTopValue = iota
tokenArrayStart
tokenArrayValue
tokenArrayComma
tokenObjectStart
tokenObjectKey
tokenObjectColon
tokenObjectValue
tokenObjectComma
)
// advance tokenstate from a separator state to a value state
func (dec *Decoder) tokenPrepareForDecode() error {
// Note: Not calling peek before switch, to avoid
// putting peek into the standard Decode path.
// peek is only called when using the Token API.
switch dec.tokenState {
case tokenArrayComma:
c, err := dec.peek()
if err != nil {
return err
}
if c != ',' {
return &SyntaxError{"expected comma after array element", 0}
}
dec.scanp++
dec.tokenState = tokenArrayValue
case tokenObjectColon:
c, err := dec.peek()
if err != nil {
return err
}
if c != ':' {
return &SyntaxError{"expected colon after object key", 0}
}
dec.scanp++
dec.tokenState = tokenObjectValue
}
return nil
}
func (dec *Decoder) tokenValueAllowed() bool {
switch dec.tokenState {
case tokenTopValue, tokenArrayStart, tokenArrayValue, tokenObjectValue:
return true
}
return false
}
func (dec *Decoder) tokenValueEnd() {
switch dec.tokenState {
case tokenArrayStart, tokenArrayValue:
dec.tokenState = tokenArrayComma
case tokenObjectValue:
dec.tokenState = tokenObjectComma
}
}
// A Delim is a JSON array or object delimiter, one of [ ] { or }.
type Delim rune
func (d Delim) String() string {
return string(d)
}
// Token returns the next JSON token in the input stream.
// At the end of the input stream, Token returns nil, io.EOF.
//
// Token guarantees that the delimiters [ ] { } it returns are
// properly nested and matched: if Token encounters an unexpected
// delimiter in the input, it will return an error.
//
// The input stream consists of basic JSON values—bool, string,
// number, and null—along with delimiters [ ] { } of type Delim
// to mark the start and end of arrays and objects.
// Commas and colons are elided.
func (dec *Decoder) Token() (Token, error) {
for {
c, err := dec.peek()
if err != nil {
return nil, err
}
switch c {
case '[':
if !dec.tokenValueAllowed() {
return dec.tokenError(c)
}
dec.scanp++
dec.tokenStack = append(dec.tokenStack, dec.tokenState)
dec.tokenState = tokenArrayStart
return Delim('['), nil
case ']':
if dec.tokenState != tokenArrayStart && dec.tokenState != tokenArrayComma {
return dec.tokenError(c)
}
dec.scanp++
dec.tokenState = dec.tokenStack[len(dec.tokenStack)-1]
dec.tokenStack = dec.tokenStack[:len(dec.tokenStack)-1]
dec.tokenValueEnd()
return Delim(']'), nil
case '{':
if !dec.tokenValueAllowed() {
return dec.tokenError(c)
}
dec.scanp++
dec.tokenStack = append(dec.tokenStack, dec.tokenState)
dec.tokenState = tokenObjectStart
return Delim('{'), nil
case '}':
if dec.tokenState != tokenObjectStart && dec.tokenState != tokenObjectComma {
return dec.tokenError(c)
}
dec.scanp++
dec.tokenState = dec.tokenStack[len(dec.tokenStack)-1]
dec.tokenStack = dec.tokenStack[:len(dec.tokenStack)-1]
dec.tokenValueEnd()
return Delim('}'), nil
case ':':
if dec.tokenState != tokenObjectColon {
return dec.tokenError(c)
}
dec.scanp++
dec.tokenState = tokenObjectValue
continue
case ',':
if dec.tokenState == tokenArrayComma {
dec.scanp++
dec.tokenState = tokenArrayValue
continue
}
if dec.tokenState == tokenObjectComma {
dec.scanp++
dec.tokenState = tokenObjectKey
continue
}
return dec.tokenError(c)
case '"':
if dec.tokenState == tokenObjectStart || dec.tokenState == tokenObjectKey {
var x string
old := dec.tokenState
dec.tokenState = tokenTopValue
err := dec.Decode(&x)
dec.tokenState = old
if err != nil {
clearOffset(err)
return nil, err
}
dec.tokenState = tokenObjectColon
return x, nil
}
fallthrough
default:
if !dec.tokenValueAllowed() {
return dec.tokenError(c)
}
var x interface{}
if err := dec.Decode(&x); err != nil {
clearOffset(err)
return nil, err
}
return x, nil
}
}
}
func clearOffset(err error) {
if s, ok := err.(*SyntaxError); ok {
s.Offset = 0
}
}
func (dec *Decoder) tokenError(c byte) (Token, error) {
var context string
switch dec.tokenState {
case tokenTopValue:
context = " looking for beginning of value"
case tokenArrayStart, tokenArrayValue, tokenObjectValue:
context = " looking for beginning of value"
case tokenArrayComma:
context = " after array element"
case tokenObjectKey:
context = " looking for beginning of object key string"
case tokenObjectColon:
context = " after object key"
case tokenObjectComma:
context = " after object key:value pair"
}
return nil, &SyntaxError{"invalid character " + quoteChar(c) + " " + context, 0}
}
// More reports whether there is another element in the
// current array or object being parsed.
func (dec *Decoder) More() bool {
c, err := dec.peek()
return err == nil && c != ']' && c != '}'
}
func (dec *Decoder) peek() (byte, error) {
var err error
for {
for i := dec.scanp; i < len(dec.buf); i++ {
c := dec.buf[i]
if isSpace(c) {
continue
}
dec.scanp = i
return c, nil
}
// buffer has been scanned, now report any error
if err != nil {
return 0, err
}
err = dec.refill()
}
}
/*
TODO
// EncodeToken writes the given JSON token to the stream.
// It returns an error if the delimiters [ ] { } are not properly used.
//
// EncodeToken does not call Flush, because usually it is part of
// a larger operation such as Encode, and those will call Flush when finished.
// Callers that create an Encoder and then invoke EncodeToken directly,
// without using Encode, need to call Flush when finished to ensure that
// the JSON is written to the underlying writer.
func (e *Encoder) EncodeToken(t Token) error {
...
}
*/
+44
View File
@@ -0,0 +1,44 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package json
import (
"strings"
)
// tagOptions is the string following a comma in a struct field's "json"
// tag, or the empty string. It does not include the leading comma.
type tagOptions string
// parseTag splits a struct field's json tag into its name and
// comma-separated options.
func parseTag(tag string) (string, tagOptions) {
if idx := strings.Index(tag, ","); idx != -1 {
return tag[:idx], tagOptions(tag[idx+1:])
}
return tag, tagOptions("")
}
// Contains reports whether a comma-separated list of options
// contains a particular substr flag. substr must be surrounded by a
// string boundary or commas.
func (o tagOptions) Contains(optionName string) bool {
if len(o) == 0 {
return false
}
s := string(o)
for s != "" {
var next string
i := strings.Index(s, ",")
if i >= 0 {
s, next = s[:i], s[i+1:]
}
if s == optionName {
return true
}
s = next
}
return false
}
+295
View File
@@ -0,0 +1,295 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jose
import (
"encoding/base64"
"fmt"
"strings"
"github.com/go-jose/go-jose/v3/json"
)
// rawJSONWebEncryption represents a raw JWE JSON object. Used for parsing/serializing.
type rawJSONWebEncryption struct {
Protected *byteBuffer `json:"protected,omitempty"`
Unprotected *rawHeader `json:"unprotected,omitempty"`
Header *rawHeader `json:"header,omitempty"`
Recipients []rawRecipientInfo `json:"recipients,omitempty"`
Aad *byteBuffer `json:"aad,omitempty"`
EncryptedKey *byteBuffer `json:"encrypted_key,omitempty"`
Iv *byteBuffer `json:"iv,omitempty"`
Ciphertext *byteBuffer `json:"ciphertext,omitempty"`
Tag *byteBuffer `json:"tag,omitempty"`
}
// rawRecipientInfo represents a raw JWE Per-Recipient header JSON object. Used for parsing/serializing.
type rawRecipientInfo struct {
Header *rawHeader `json:"header,omitempty"`
EncryptedKey string `json:"encrypted_key,omitempty"`
}
// JSONWebEncryption represents an encrypted JWE object after parsing.
type JSONWebEncryption struct {
Header Header
protected, unprotected *rawHeader
recipients []recipientInfo
aad, iv, ciphertext, tag []byte
original *rawJSONWebEncryption
}
// recipientInfo represents a raw JWE Per-Recipient header JSON object after parsing.
type recipientInfo struct {
header *rawHeader
encryptedKey []byte
}
// GetAuthData retrieves the (optional) authenticated data attached to the object.
func (obj JSONWebEncryption) GetAuthData() []byte {
if obj.aad != nil {
out := make([]byte, len(obj.aad))
copy(out, obj.aad)
return out
}
return nil
}
// Get the merged header values
func (obj JSONWebEncryption) mergedHeaders(recipient *recipientInfo) rawHeader {
out := rawHeader{}
out.merge(obj.protected)
out.merge(obj.unprotected)
if recipient != nil {
out.merge(recipient.header)
}
return out
}
// Get the additional authenticated data from a JWE object.
func (obj JSONWebEncryption) computeAuthData() []byte {
var protected string
switch {
case obj.original != nil && obj.original.Protected != nil:
protected = obj.original.Protected.base64()
case obj.protected != nil:
protected = base64.RawURLEncoding.EncodeToString(mustSerializeJSON((obj.protected)))
default:
protected = ""
}
output := []byte(protected)
if obj.aad != nil {
output = append(output, '.')
output = append(output, []byte(base64.RawURLEncoding.EncodeToString(obj.aad))...)
}
return output
}
// ParseEncrypted parses an encrypted message in compact or JWE JSON Serialization format.
func ParseEncrypted(input string) (*JSONWebEncryption, error) {
input = stripWhitespace(input)
if strings.HasPrefix(input, "{") {
return parseEncryptedFull(input)
}
return parseEncryptedCompact(input)
}
// parseEncryptedFull parses a message in compact format.
func parseEncryptedFull(input string) (*JSONWebEncryption, error) {
var parsed rawJSONWebEncryption
err := json.Unmarshal([]byte(input), &parsed)
if err != nil {
return nil, err
}
return parsed.sanitized()
}
// sanitized produces a cleaned-up JWE object from the raw JSON.
func (parsed *rawJSONWebEncryption) sanitized() (*JSONWebEncryption, error) {
obj := &JSONWebEncryption{
original: parsed,
unprotected: parsed.Unprotected,
}
// Check that there is not a nonce in the unprotected headers
if parsed.Unprotected != nil {
if nonce := parsed.Unprotected.getNonce(); nonce != "" {
return nil, ErrUnprotectedNonce
}
}
if parsed.Header != nil {
if nonce := parsed.Header.getNonce(); nonce != "" {
return nil, ErrUnprotectedNonce
}
}
if parsed.Protected != nil && len(parsed.Protected.bytes()) > 0 {
err := json.Unmarshal(parsed.Protected.bytes(), &obj.protected)
if err != nil {
return nil, fmt.Errorf("go-jose/go-jose: invalid protected header: %s, %s", err, parsed.Protected.base64())
}
}
// Note: this must be called _after_ we parse the protected header,
// otherwise fields from the protected header will not get picked up.
var err error
mergedHeaders := obj.mergedHeaders(nil)
obj.Header, err = mergedHeaders.sanitized()
if err != nil {
return nil, fmt.Errorf("go-jose/go-jose: cannot sanitize merged headers: %v (%v)", err, mergedHeaders)
}
if len(parsed.Recipients) == 0 {
obj.recipients = []recipientInfo{
{
header: parsed.Header,
encryptedKey: parsed.EncryptedKey.bytes(),
},
}
} else {
obj.recipients = make([]recipientInfo, len(parsed.Recipients))
for r := range parsed.Recipients {
encryptedKey, err := base64URLDecode(parsed.Recipients[r].EncryptedKey)
if err != nil {
return nil, err
}
// Check that there is not a nonce in the unprotected header
if parsed.Recipients[r].Header != nil && parsed.Recipients[r].Header.getNonce() != "" {
return nil, ErrUnprotectedNonce
}
obj.recipients[r].header = parsed.Recipients[r].Header
obj.recipients[r].encryptedKey = encryptedKey
}
}
for _, recipient := range obj.recipients {
headers := obj.mergedHeaders(&recipient)
if headers.getAlgorithm() == "" || headers.getEncryption() == "" {
return nil, fmt.Errorf("go-jose/go-jose: message is missing alg/enc headers")
}
}
obj.iv = parsed.Iv.bytes()
obj.ciphertext = parsed.Ciphertext.bytes()
obj.tag = parsed.Tag.bytes()
obj.aad = parsed.Aad.bytes()
return obj, nil
}
// parseEncryptedCompact parses a message in compact format.
func parseEncryptedCompact(input string) (*JSONWebEncryption, error) {
parts := strings.Split(input, ".")
if len(parts) != 5 {
return nil, fmt.Errorf("go-jose/go-jose: compact JWE format must have five parts")
}
rawProtected, err := base64URLDecode(parts[0])
if err != nil {
return nil, err
}
encryptedKey, err := base64URLDecode(parts[1])
if err != nil {
return nil, err
}
iv, err := base64URLDecode(parts[2])
if err != nil {
return nil, err
}
ciphertext, err := base64URLDecode(parts[3])
if err != nil {
return nil, err
}
tag, err := base64URLDecode(parts[4])
if err != nil {
return nil, err
}
raw := &rawJSONWebEncryption{
Protected: newBuffer(rawProtected),
EncryptedKey: newBuffer(encryptedKey),
Iv: newBuffer(iv),
Ciphertext: newBuffer(ciphertext),
Tag: newBuffer(tag),
}
return raw.sanitized()
}
// CompactSerialize serializes an object using the compact serialization format.
func (obj JSONWebEncryption) CompactSerialize() (string, error) {
if len(obj.recipients) != 1 || obj.unprotected != nil ||
obj.protected == nil || obj.recipients[0].header != nil {
return "", ErrNotSupported
}
serializedProtected := mustSerializeJSON(obj.protected)
return fmt.Sprintf(
"%s.%s.%s.%s.%s",
base64.RawURLEncoding.EncodeToString(serializedProtected),
base64.RawURLEncoding.EncodeToString(obj.recipients[0].encryptedKey),
base64.RawURLEncoding.EncodeToString(obj.iv),
base64.RawURLEncoding.EncodeToString(obj.ciphertext),
base64.RawURLEncoding.EncodeToString(obj.tag)), nil
}
// FullSerialize serializes an object using the full JSON serialization format.
func (obj JSONWebEncryption) FullSerialize() string {
raw := rawJSONWebEncryption{
Unprotected: obj.unprotected,
Iv: newBuffer(obj.iv),
Ciphertext: newBuffer(obj.ciphertext),
EncryptedKey: newBuffer(obj.recipients[0].encryptedKey),
Tag: newBuffer(obj.tag),
Aad: newBuffer(obj.aad),
Recipients: []rawRecipientInfo{},
}
if len(obj.recipients) > 1 {
for _, recipient := range obj.recipients {
info := rawRecipientInfo{
Header: recipient.header,
EncryptedKey: base64.RawURLEncoding.EncodeToString(recipient.encryptedKey),
}
raw.Recipients = append(raw.Recipients, info)
}
} else {
// Use flattened serialization
raw.Header = obj.recipients[0].header
raw.EncryptedKey = newBuffer(obj.recipients[0].encryptedKey)
}
if obj.protected != nil {
raw.Protected = newBuffer(mustSerializeJSON(obj.protected))
}
return string(mustSerializeJSON(raw))
}
+798
View File
@@ -0,0 +1,798 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jose
import (
"bytes"
"crypto"
"crypto/ecdsa"
"crypto/ed25519"
"crypto/elliptic"
"crypto/rsa"
"crypto/sha1"
"crypto/sha256"
"crypto/x509"
"encoding/base64"
"encoding/hex"
"errors"
"fmt"
"math/big"
"net/url"
"reflect"
"strings"
"github.com/go-jose/go-jose/v3/json"
)
// rawJSONWebKey represents a public or private key in JWK format, used for parsing/serializing.
type rawJSONWebKey struct {
Use string `json:"use,omitempty"`
Kty string `json:"kty,omitempty"`
Kid string `json:"kid,omitempty"`
Crv string `json:"crv,omitempty"`
Alg string `json:"alg,omitempty"`
K *byteBuffer `json:"k,omitempty"`
X *byteBuffer `json:"x,omitempty"`
Y *byteBuffer `json:"y,omitempty"`
N *byteBuffer `json:"n,omitempty"`
E *byteBuffer `json:"e,omitempty"`
// -- Following fields are only used for private keys --
// RSA uses D, P and Q, while ECDSA uses only D. Fields Dp, Dq, and Qi are
// completely optional. Therefore for RSA/ECDSA, D != nil is a contract that
// we have a private key whereas D == nil means we have only a public key.
D *byteBuffer `json:"d,omitempty"`
P *byteBuffer `json:"p,omitempty"`
Q *byteBuffer `json:"q,omitempty"`
Dp *byteBuffer `json:"dp,omitempty"`
Dq *byteBuffer `json:"dq,omitempty"`
Qi *byteBuffer `json:"qi,omitempty"`
// Certificates
X5c []string `json:"x5c,omitempty"`
X5u string `json:"x5u,omitempty"`
X5tSHA1 string `json:"x5t,omitempty"`
X5tSHA256 string `json:"x5t#S256,omitempty"`
}
// JSONWebKey represents a public or private key in JWK format.
type JSONWebKey struct {
// Cryptographic key, can be a symmetric or asymmetric key.
Key interface{}
// Key identifier, parsed from `kid` header.
KeyID string
// Key algorithm, parsed from `alg` header.
Algorithm string
// Key use, parsed from `use` header.
Use string
// X.509 certificate chain, parsed from `x5c` header.
Certificates []*x509.Certificate
// X.509 certificate URL, parsed from `x5u` header.
CertificatesURL *url.URL
// X.509 certificate thumbprint (SHA-1), parsed from `x5t` header.
CertificateThumbprintSHA1 []byte
// X.509 certificate thumbprint (SHA-256), parsed from `x5t#S256` header.
CertificateThumbprintSHA256 []byte
}
// MarshalJSON serializes the given key to its JSON representation.
func (k JSONWebKey) MarshalJSON() ([]byte, error) {
var raw *rawJSONWebKey
var err error
switch key := k.Key.(type) {
case ed25519.PublicKey:
raw = fromEdPublicKey(key)
case *ecdsa.PublicKey:
raw, err = fromEcPublicKey(key)
case *rsa.PublicKey:
raw = fromRsaPublicKey(key)
case ed25519.PrivateKey:
raw, err = fromEdPrivateKey(key)
case *ecdsa.PrivateKey:
raw, err = fromEcPrivateKey(key)
case *rsa.PrivateKey:
raw, err = fromRsaPrivateKey(key)
case []byte:
raw, err = fromSymmetricKey(key)
default:
return nil, fmt.Errorf("go-jose/go-jose: unknown key type '%s'", reflect.TypeOf(key))
}
if err != nil {
return nil, err
}
raw.Kid = k.KeyID
raw.Alg = k.Algorithm
raw.Use = k.Use
for _, cert := range k.Certificates {
raw.X5c = append(raw.X5c, base64.StdEncoding.EncodeToString(cert.Raw))
}
x5tSHA1Len := len(k.CertificateThumbprintSHA1)
x5tSHA256Len := len(k.CertificateThumbprintSHA256)
if x5tSHA1Len > 0 {
if x5tSHA1Len != sha1.Size {
return nil, fmt.Errorf("go-jose/go-jose: invalid SHA-1 thumbprint (must be %d bytes, not %d)", sha1.Size, x5tSHA1Len)
}
raw.X5tSHA1 = base64.RawURLEncoding.EncodeToString(k.CertificateThumbprintSHA1)
}
if x5tSHA256Len > 0 {
if x5tSHA256Len != sha256.Size {
return nil, fmt.Errorf("go-jose/go-jose: invalid SHA-256 thumbprint (must be %d bytes, not %d)", sha256.Size, x5tSHA256Len)
}
raw.X5tSHA256 = base64.RawURLEncoding.EncodeToString(k.CertificateThumbprintSHA256)
}
// If cert chain is attached (as opposed to being behind a URL), check the
// keys thumbprints to make sure they match what is expected. This is to
// ensure we don't accidentally produce a JWK with semantically inconsistent
// data in the headers.
if len(k.Certificates) > 0 {
expectedSHA1 := sha1.Sum(k.Certificates[0].Raw)
expectedSHA256 := sha256.Sum256(k.Certificates[0].Raw)
if len(k.CertificateThumbprintSHA1) > 0 && !bytes.Equal(k.CertificateThumbprintSHA1, expectedSHA1[:]) {
return nil, errors.New("go-jose/go-jose: invalid SHA-1 thumbprint, does not match cert chain")
}
if len(k.CertificateThumbprintSHA256) > 0 && !bytes.Equal(k.CertificateThumbprintSHA256, expectedSHA256[:]) {
return nil, errors.New("go-jose/go-jose: invalid or SHA-256 thumbprint, does not match cert chain")
}
}
if k.CertificatesURL != nil {
raw.X5u = k.CertificatesURL.String()
}
return json.Marshal(raw)
}
// UnmarshalJSON reads a key from its JSON representation.
func (k *JSONWebKey) UnmarshalJSON(data []byte) (err error) {
var raw rawJSONWebKey
err = json.Unmarshal(data, &raw)
if err != nil {
return err
}
certs, err := parseCertificateChain(raw.X5c)
if err != nil {
return fmt.Errorf("go-jose/go-jose: failed to unmarshal x5c field: %s", err)
}
var key interface{}
var certPub interface{}
var keyPub interface{}
if len(certs) > 0 {
// We need to check that leaf public key matches the key embedded in this
// JWK, as required by the standard (see RFC 7517, Section 4.7). Otherwise
// the JWK parsed could be semantically invalid. Technically, should also
// check key usage fields and other extensions on the cert here, but the
// standard doesn't exactly explain how they're supposed to map from the
// JWK representation to the X.509 extensions.
certPub = certs[0].PublicKey
}
switch raw.Kty {
case "EC":
if raw.D != nil {
key, err = raw.ecPrivateKey()
if err == nil {
keyPub = key.(*ecdsa.PrivateKey).Public()
}
} else {
key, err = raw.ecPublicKey()
keyPub = key
}
case "RSA":
if raw.D != nil {
key, err = raw.rsaPrivateKey()
if err == nil {
keyPub = key.(*rsa.PrivateKey).Public()
}
} else {
key, err = raw.rsaPublicKey()
keyPub = key
}
case "oct":
if certPub != nil {
return errors.New("go-jose/go-jose: invalid JWK, found 'oct' (symmetric) key with cert chain")
}
key, err = raw.symmetricKey()
case "OKP":
if raw.Crv == "Ed25519" && raw.X != nil {
if raw.D != nil {
key, err = raw.edPrivateKey()
if err == nil {
keyPub = key.(ed25519.PrivateKey).Public()
}
} else {
key, err = raw.edPublicKey()
keyPub = key
}
} else {
err = fmt.Errorf("go-jose/go-jose: unknown curve %s'", raw.Crv)
}
default:
err = fmt.Errorf("go-jose/go-jose: unknown json web key type '%s'", raw.Kty)
}
if err != nil {
return
}
if certPub != nil && keyPub != nil {
if !reflect.DeepEqual(certPub, keyPub) {
return errors.New("go-jose/go-jose: invalid JWK, public keys in key and x5c fields do not match")
}
}
*k = JSONWebKey{Key: key, KeyID: raw.Kid, Algorithm: raw.Alg, Use: raw.Use, Certificates: certs}
if raw.X5u != "" {
k.CertificatesURL, err = url.Parse(raw.X5u)
if err != nil {
return fmt.Errorf("go-jose/go-jose: invalid JWK, x5u header is invalid URL: %w", err)
}
}
// x5t parameters are base64url-encoded SHA thumbprints
// See RFC 7517, Section 4.8, https://tools.ietf.org/html/rfc7517#section-4.8
x5tSHA1bytes, err := base64URLDecode(raw.X5tSHA1)
if err != nil {
return errors.New("go-jose/go-jose: invalid JWK, x5t header has invalid encoding")
}
// RFC 7517, Section 4.8 is ambiguous as to whether the digest output should be byte or hex,
// for this reason, after base64 decoding, if the size is sha1.Size it's likely that the value is a byte encoded
// checksum so we skip this. Otherwise if the checksum was hex encoded we expect a 40 byte sized array so we'll
// try to hex decode it. When Marshalling this value we'll always use a base64 encoded version of byte format checksum.
if len(x5tSHA1bytes) == 2*sha1.Size {
hx, err := hex.DecodeString(string(x5tSHA1bytes))
if err != nil {
return fmt.Errorf("go-jose/go-jose: invalid JWK, unable to hex decode x5t: %v", err)
}
x5tSHA1bytes = hx
}
k.CertificateThumbprintSHA1 = x5tSHA1bytes
x5tSHA256bytes, err := base64URLDecode(raw.X5tSHA256)
if err != nil {
return errors.New("go-jose/go-jose: invalid JWK, x5t#S256 header has invalid encoding")
}
if len(x5tSHA256bytes) == 2*sha256.Size {
hx256, err := hex.DecodeString(string(x5tSHA256bytes))
if err != nil {
return fmt.Errorf("go-jose/go-jose: invalid JWK, unable to hex decode x5t#S256: %v", err)
}
x5tSHA256bytes = hx256
}
k.CertificateThumbprintSHA256 = x5tSHA256bytes
x5tSHA1Len := len(k.CertificateThumbprintSHA1)
x5tSHA256Len := len(k.CertificateThumbprintSHA256)
if x5tSHA1Len > 0 && x5tSHA1Len != sha1.Size {
return errors.New("go-jose/go-jose: invalid JWK, x5t header is of incorrect size")
}
if x5tSHA256Len > 0 && x5tSHA256Len != sha256.Size {
return errors.New("go-jose/go-jose: invalid JWK, x5t#S256 header is of incorrect size")
}
// If certificate chain *and* thumbprints are set, verify correctness.
if len(k.Certificates) > 0 {
leaf := k.Certificates[0]
sha1sum := sha1.Sum(leaf.Raw)
sha256sum := sha256.Sum256(leaf.Raw)
if len(k.CertificateThumbprintSHA1) > 0 && !bytes.Equal(sha1sum[:], k.CertificateThumbprintSHA1) {
return errors.New("go-jose/go-jose: invalid JWK, x5c thumbprint does not match x5t value")
}
if len(k.CertificateThumbprintSHA256) > 0 && !bytes.Equal(sha256sum[:], k.CertificateThumbprintSHA256) {
return errors.New("go-jose/go-jose: invalid JWK, x5c thumbprint does not match x5t#S256 value")
}
}
return
}
// JSONWebKeySet represents a JWK Set object.
type JSONWebKeySet struct {
Keys []JSONWebKey `json:"keys"`
}
// Key convenience method returns keys by key ID. Specification states
// that a JWK Set "SHOULD" use distinct key IDs, but allows for some
// cases where they are not distinct. Hence method returns a slice
// of JSONWebKeys.
func (s *JSONWebKeySet) Key(kid string) []JSONWebKey {
var keys []JSONWebKey
for _, key := range s.Keys {
if key.KeyID == kid {
keys = append(keys, key)
}
}
return keys
}
const rsaThumbprintTemplate = `{"e":"%s","kty":"RSA","n":"%s"}`
const ecThumbprintTemplate = `{"crv":"%s","kty":"EC","x":"%s","y":"%s"}`
const edThumbprintTemplate = `{"crv":"%s","kty":"OKP","x":"%s"}`
func ecThumbprintInput(curve elliptic.Curve, x, y *big.Int) (string, error) {
coordLength := curveSize(curve)
crv, err := curveName(curve)
if err != nil {
return "", err
}
if len(x.Bytes()) > coordLength || len(y.Bytes()) > coordLength {
return "", errors.New("go-jose/go-jose: invalid elliptic key (too large)")
}
return fmt.Sprintf(ecThumbprintTemplate, crv,
newFixedSizeBuffer(x.Bytes(), coordLength).base64(),
newFixedSizeBuffer(y.Bytes(), coordLength).base64()), nil
}
func rsaThumbprintInput(n *big.Int, e int) (string, error) {
return fmt.Sprintf(rsaThumbprintTemplate,
newBufferFromInt(uint64(e)).base64(),
newBuffer(n.Bytes()).base64()), nil
}
func edThumbprintInput(ed ed25519.PublicKey) (string, error) {
crv := "Ed25519"
if len(ed) > 32 {
return "", errors.New("go-jose/go-jose: invalid elliptic key (too large)")
}
return fmt.Sprintf(edThumbprintTemplate, crv,
newFixedSizeBuffer(ed, 32).base64()), nil
}
// Thumbprint computes the JWK Thumbprint of a key using the
// indicated hash algorithm.
func (k *JSONWebKey) Thumbprint(hash crypto.Hash) ([]byte, error) {
var input string
var err error
switch key := k.Key.(type) {
case ed25519.PublicKey:
input, err = edThumbprintInput(key)
case *ecdsa.PublicKey:
input, err = ecThumbprintInput(key.Curve, key.X, key.Y)
case *ecdsa.PrivateKey:
input, err = ecThumbprintInput(key.Curve, key.X, key.Y)
case *rsa.PublicKey:
input, err = rsaThumbprintInput(key.N, key.E)
case *rsa.PrivateKey:
input, err = rsaThumbprintInput(key.N, key.E)
case ed25519.PrivateKey:
input, err = edThumbprintInput(ed25519.PublicKey(key[32:]))
default:
return nil, fmt.Errorf("go-jose/go-jose: unknown key type '%s'", reflect.TypeOf(key))
}
if err != nil {
return nil, err
}
h := hash.New()
_, _ = h.Write([]byte(input))
return h.Sum(nil), nil
}
// IsPublic returns true if the JWK represents a public key (not symmetric, not private).
func (k *JSONWebKey) IsPublic() bool {
switch k.Key.(type) {
case *ecdsa.PublicKey, *rsa.PublicKey, ed25519.PublicKey:
return true
default:
return false
}
}
// Public creates JSONWebKey with corresponding public key if JWK represents asymmetric private key.
func (k *JSONWebKey) Public() JSONWebKey {
if k.IsPublic() {
return *k
}
ret := *k
switch key := k.Key.(type) {
case *ecdsa.PrivateKey:
ret.Key = key.Public()
case *rsa.PrivateKey:
ret.Key = key.Public()
case ed25519.PrivateKey:
ret.Key = key.Public()
default:
return JSONWebKey{} // returning invalid key
}
return ret
}
// Valid checks that the key contains the expected parameters.
func (k *JSONWebKey) Valid() bool {
if k.Key == nil {
return false
}
switch key := k.Key.(type) {
case *ecdsa.PublicKey:
if key.Curve == nil || key.X == nil || key.Y == nil {
return false
}
case *ecdsa.PrivateKey:
if key.Curve == nil || key.X == nil || key.Y == nil || key.D == nil {
return false
}
case *rsa.PublicKey:
if key.N == nil || key.E == 0 {
return false
}
case *rsa.PrivateKey:
if key.N == nil || key.E == 0 || key.D == nil || len(key.Primes) < 2 {
return false
}
case ed25519.PublicKey:
if len(key) != 32 {
return false
}
case ed25519.PrivateKey:
if len(key) != 64 {
return false
}
default:
return false
}
return true
}
func (key rawJSONWebKey) rsaPublicKey() (*rsa.PublicKey, error) {
if key.N == nil || key.E == nil {
return nil, fmt.Errorf("go-jose/go-jose: invalid RSA key, missing n/e values")
}
return &rsa.PublicKey{
N: key.N.bigInt(),
E: key.E.toInt(),
}, nil
}
func fromEdPublicKey(pub ed25519.PublicKey) *rawJSONWebKey {
return &rawJSONWebKey{
Kty: "OKP",
Crv: "Ed25519",
X: newBuffer(pub),
}
}
func fromRsaPublicKey(pub *rsa.PublicKey) *rawJSONWebKey {
return &rawJSONWebKey{
Kty: "RSA",
N: newBuffer(pub.N.Bytes()),
E: newBufferFromInt(uint64(pub.E)),
}
}
func (key rawJSONWebKey) ecPublicKey() (*ecdsa.PublicKey, error) {
var curve elliptic.Curve
switch key.Crv {
case "P-256":
curve = elliptic.P256()
case "P-384":
curve = elliptic.P384()
case "P-521":
curve = elliptic.P521()
default:
return nil, fmt.Errorf("go-jose/go-jose: unsupported elliptic curve '%s'", key.Crv)
}
if key.X == nil || key.Y == nil {
return nil, errors.New("go-jose/go-jose: invalid EC key, missing x/y values")
}
// The length of this octet string MUST be the full size of a coordinate for
// the curve specified in the "crv" parameter.
// https://tools.ietf.org/html/rfc7518#section-6.2.1.2
if curveSize(curve) != len(key.X.data) {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC public key, wrong length for x")
}
if curveSize(curve) != len(key.Y.data) {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC public key, wrong length for y")
}
x := key.X.bigInt()
y := key.Y.bigInt()
if !curve.IsOnCurve(x, y) {
return nil, errors.New("go-jose/go-jose: invalid EC key, X/Y are not on declared curve")
}
return &ecdsa.PublicKey{
Curve: curve,
X: x,
Y: y,
}, nil
}
func fromEcPublicKey(pub *ecdsa.PublicKey) (*rawJSONWebKey, error) {
if pub == nil || pub.X == nil || pub.Y == nil {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC key (nil, or X/Y missing)")
}
name, err := curveName(pub.Curve)
if err != nil {
return nil, err
}
size := curveSize(pub.Curve)
xBytes := pub.X.Bytes()
yBytes := pub.Y.Bytes()
if len(xBytes) > size || len(yBytes) > size {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC key (X/Y too large)")
}
key := &rawJSONWebKey{
Kty: "EC",
Crv: name,
X: newFixedSizeBuffer(xBytes, size),
Y: newFixedSizeBuffer(yBytes, size),
}
return key, nil
}
func (key rawJSONWebKey) edPrivateKey() (ed25519.PrivateKey, error) {
var missing []string
switch {
case key.D == nil:
missing = append(missing, "D")
case key.X == nil:
missing = append(missing, "X")
}
if len(missing) > 0 {
return nil, fmt.Errorf("go-jose/go-jose: invalid Ed25519 private key, missing %s value(s)", strings.Join(missing, ", "))
}
privateKey := make([]byte, ed25519.PrivateKeySize)
copy(privateKey[0:32], key.D.bytes())
copy(privateKey[32:], key.X.bytes())
rv := ed25519.PrivateKey(privateKey)
return rv, nil
}
func (key rawJSONWebKey) edPublicKey() (ed25519.PublicKey, error) {
if key.X == nil {
return nil, fmt.Errorf("go-jose/go-jose: invalid Ed key, missing x value")
}
publicKey := make([]byte, ed25519.PublicKeySize)
copy(publicKey[0:32], key.X.bytes())
rv := ed25519.PublicKey(publicKey)
return rv, nil
}
func (key rawJSONWebKey) rsaPrivateKey() (*rsa.PrivateKey, error) {
var missing []string
switch {
case key.N == nil:
missing = append(missing, "N")
case key.E == nil:
missing = append(missing, "E")
case key.D == nil:
missing = append(missing, "D")
case key.P == nil:
missing = append(missing, "P")
case key.Q == nil:
missing = append(missing, "Q")
}
if len(missing) > 0 {
return nil, fmt.Errorf("go-jose/go-jose: invalid RSA private key, missing %s value(s)", strings.Join(missing, ", "))
}
rv := &rsa.PrivateKey{
PublicKey: rsa.PublicKey{
N: key.N.bigInt(),
E: key.E.toInt(),
},
D: key.D.bigInt(),
Primes: []*big.Int{
key.P.bigInt(),
key.Q.bigInt(),
},
}
if key.Dp != nil {
rv.Precomputed.Dp = key.Dp.bigInt()
}
if key.Dq != nil {
rv.Precomputed.Dq = key.Dq.bigInt()
}
if key.Qi != nil {
rv.Precomputed.Qinv = key.Qi.bigInt()
}
err := rv.Validate()
return rv, err
}
func fromEdPrivateKey(ed ed25519.PrivateKey) (*rawJSONWebKey, error) {
raw := fromEdPublicKey(ed25519.PublicKey(ed[32:]))
raw.D = newBuffer(ed[0:32])
return raw, nil
}
func fromRsaPrivateKey(rsa *rsa.PrivateKey) (*rawJSONWebKey, error) {
if len(rsa.Primes) != 2 {
return nil, ErrUnsupportedKeyType
}
raw := fromRsaPublicKey(&rsa.PublicKey)
raw.D = newBuffer(rsa.D.Bytes())
raw.P = newBuffer(rsa.Primes[0].Bytes())
raw.Q = newBuffer(rsa.Primes[1].Bytes())
if rsa.Precomputed.Dp != nil {
raw.Dp = newBuffer(rsa.Precomputed.Dp.Bytes())
}
if rsa.Precomputed.Dq != nil {
raw.Dq = newBuffer(rsa.Precomputed.Dq.Bytes())
}
if rsa.Precomputed.Qinv != nil {
raw.Qi = newBuffer(rsa.Precomputed.Qinv.Bytes())
}
return raw, nil
}
func (key rawJSONWebKey) ecPrivateKey() (*ecdsa.PrivateKey, error) {
var curve elliptic.Curve
switch key.Crv {
case "P-256":
curve = elliptic.P256()
case "P-384":
curve = elliptic.P384()
case "P-521":
curve = elliptic.P521()
default:
return nil, fmt.Errorf("go-jose/go-jose: unsupported elliptic curve '%s'", key.Crv)
}
if key.X == nil || key.Y == nil || key.D == nil {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC private key, missing x/y/d values")
}
// The length of this octet string MUST be the full size of a coordinate for
// the curve specified in the "crv" parameter.
// https://tools.ietf.org/html/rfc7518#section-6.2.1.2
if curveSize(curve) != len(key.X.data) {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC private key, wrong length for x")
}
if curveSize(curve) != len(key.Y.data) {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC private key, wrong length for y")
}
// https://tools.ietf.org/html/rfc7518#section-6.2.2.1
if dSize(curve) != len(key.D.data) {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC private key, wrong length for d")
}
x := key.X.bigInt()
y := key.Y.bigInt()
if !curve.IsOnCurve(x, y) {
return nil, errors.New("go-jose/go-jose: invalid EC key, X/Y are not on declared curve")
}
return &ecdsa.PrivateKey{
PublicKey: ecdsa.PublicKey{
Curve: curve,
X: x,
Y: y,
},
D: key.D.bigInt(),
}, nil
}
func fromEcPrivateKey(ec *ecdsa.PrivateKey) (*rawJSONWebKey, error) {
raw, err := fromEcPublicKey(&ec.PublicKey)
if err != nil {
return nil, err
}
if ec.D == nil {
return nil, fmt.Errorf("go-jose/go-jose: invalid EC private key")
}
raw.D = newFixedSizeBuffer(ec.D.Bytes(), dSize(ec.PublicKey.Curve))
return raw, nil
}
// dSize returns the size in octets for the "d" member of an elliptic curve
// private key.
// The length of this octet string MUST be ceiling(log-base-2(n)/8)
// octets (where n is the order of the curve).
// https://tools.ietf.org/html/rfc7518#section-6.2.2.1
func dSize(curve elliptic.Curve) int {
order := curve.Params().P
bitLen := order.BitLen()
size := bitLen / 8
if bitLen%8 != 0 {
size++
}
return size
}
func fromSymmetricKey(key []byte) (*rawJSONWebKey, error) {
return &rawJSONWebKey{
Kty: "oct",
K: newBuffer(key),
}, nil
}
func (key rawJSONWebKey) symmetricKey() ([]byte, error) {
if key.K == nil {
return nil, fmt.Errorf("go-jose/go-jose: invalid OCT (symmetric) key, missing k value")
}
return key.K.bytes(), nil
}
func tryJWKS(key interface{}, headers ...Header) interface{} {
var jwks JSONWebKeySet
switch jwksType := key.(type) {
case *JSONWebKeySet:
jwks = *jwksType
case JSONWebKeySet:
jwks = jwksType
default:
return key
}
var kid string
for _, header := range headers {
if header.KeyID != "" {
kid = header.KeyID
break
}
}
if kid == "" {
return key
}
keys := jwks.Key(kid)
if len(keys) == 0 {
return key
}
return keys[0].Key
}
+366
View File
@@ -0,0 +1,366 @@
/*-
* Copyright 2014 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jose
import (
"bytes"
"encoding/base64"
"errors"
"fmt"
"strings"
"github.com/go-jose/go-jose/v3/json"
)
// rawJSONWebSignature represents a raw JWS JSON object. Used for parsing/serializing.
type rawJSONWebSignature struct {
Payload *byteBuffer `json:"payload,omitempty"`
Signatures []rawSignatureInfo `json:"signatures,omitempty"`
Protected *byteBuffer `json:"protected,omitempty"`
Header *rawHeader `json:"header,omitempty"`
Signature *byteBuffer `json:"signature,omitempty"`
}
// rawSignatureInfo represents a single JWS signature over the JWS payload and protected header.
type rawSignatureInfo struct {
Protected *byteBuffer `json:"protected,omitempty"`
Header *rawHeader `json:"header,omitempty"`
Signature *byteBuffer `json:"signature,omitempty"`
}
// JSONWebSignature represents a signed JWS object after parsing.
type JSONWebSignature struct {
payload []byte
// Signatures attached to this object (may be more than one for multi-sig).
// Be careful about accessing these directly, prefer to use Verify() or
// VerifyMulti() to ensure that the data you're getting is verified.
Signatures []Signature
}
// Signature represents a single signature over the JWS payload and protected header.
type Signature struct {
// Merged header fields. Contains both protected and unprotected header
// values. Prefer using Protected and Unprotected fields instead of this.
// Values in this header may or may not have been signed and in general
// should not be trusted.
Header Header
// Protected header. Values in this header were signed and
// will be verified as part of the signature verification process.
Protected Header
// Unprotected header. Values in this header were not signed
// and in general should not be trusted.
Unprotected Header
// The actual signature value
Signature []byte
protected *rawHeader
header *rawHeader
original *rawSignatureInfo
}
// ParseSigned parses a signed message in compact or JWS JSON Serialization format.
func ParseSigned(signature string) (*JSONWebSignature, error) {
signature = stripWhitespace(signature)
if strings.HasPrefix(signature, "{") {
return parseSignedFull(signature)
}
return parseSignedCompact(signature, nil)
}
// ParseDetached parses a signed message in compact serialization format with detached payload.
func ParseDetached(signature string, payload []byte) (*JSONWebSignature, error) {
if payload == nil {
return nil, errors.New("go-jose/go-jose: nil payload")
}
return parseSignedCompact(stripWhitespace(signature), payload)
}
// Get a header value
func (sig Signature) mergedHeaders() rawHeader {
out := rawHeader{}
out.merge(sig.protected)
out.merge(sig.header)
return out
}
// Compute data to be signed
func (obj JSONWebSignature) computeAuthData(payload []byte, signature *Signature) ([]byte, error) {
var authData bytes.Buffer
protectedHeader := new(rawHeader)
if signature.original != nil && signature.original.Protected != nil {
if err := json.Unmarshal(signature.original.Protected.bytes(), protectedHeader); err != nil {
return nil, err
}
authData.WriteString(signature.original.Protected.base64())
} else if signature.protected != nil {
protectedHeader = signature.protected
authData.WriteString(base64.RawURLEncoding.EncodeToString(mustSerializeJSON(protectedHeader)))
}
needsBase64 := true
if protectedHeader != nil {
var err error
if needsBase64, err = protectedHeader.getB64(); err != nil {
needsBase64 = true
}
}
authData.WriteByte('.')
if needsBase64 {
authData.WriteString(base64.RawURLEncoding.EncodeToString(payload))
} else {
authData.Write(payload)
}
return authData.Bytes(), nil
}
// parseSignedFull parses a message in full format.
func parseSignedFull(input string) (*JSONWebSignature, error) {
var parsed rawJSONWebSignature
err := json.Unmarshal([]byte(input), &parsed)
if err != nil {
return nil, err
}
return parsed.sanitized()
}
// sanitized produces a cleaned-up JWS object from the raw JSON.
func (parsed *rawJSONWebSignature) sanitized() (*JSONWebSignature, error) {
if parsed.Payload == nil {
return nil, fmt.Errorf("go-jose/go-jose: missing payload in JWS message")
}
obj := &JSONWebSignature{
payload: parsed.Payload.bytes(),
Signatures: make([]Signature, len(parsed.Signatures)),
}
if len(parsed.Signatures) == 0 {
// No signatures array, must be flattened serialization
signature := Signature{}
if parsed.Protected != nil && len(parsed.Protected.bytes()) > 0 {
signature.protected = &rawHeader{}
err := json.Unmarshal(parsed.Protected.bytes(), signature.protected)
if err != nil {
return nil, err
}
}
// Check that there is not a nonce in the unprotected header
if parsed.Header != nil && parsed.Header.getNonce() != "" {
return nil, ErrUnprotectedNonce
}
signature.header = parsed.Header
signature.Signature = parsed.Signature.bytes()
// Make a fake "original" rawSignatureInfo to store the unprocessed
// Protected header. This is necessary because the Protected header can
// contain arbitrary fields not registered as part of the spec. See
// https://tools.ietf.org/html/draft-ietf-jose-json-web-signature-41#section-4
// If we unmarshal Protected into a rawHeader with its explicit list of fields,
// we cannot marshal losslessly. So we have to keep around the original bytes.
// This is used in computeAuthData, which will first attempt to use
// the original bytes of a protected header, and fall back on marshaling the
// header struct only if those bytes are not available.
signature.original = &rawSignatureInfo{
Protected: parsed.Protected,
Header: parsed.Header,
Signature: parsed.Signature,
}
var err error
signature.Header, err = signature.mergedHeaders().sanitized()
if err != nil {
return nil, err
}
if signature.header != nil {
signature.Unprotected, err = signature.header.sanitized()
if err != nil {
return nil, err
}
}
if signature.protected != nil {
signature.Protected, err = signature.protected.sanitized()
if err != nil {
return nil, err
}
}
// As per RFC 7515 Section 4.1.3, only public keys are allowed to be embedded.
jwk := signature.Header.JSONWebKey
if jwk != nil && (!jwk.Valid() || !jwk.IsPublic()) {
return nil, errors.New("go-jose/go-jose: invalid embedded jwk, must be public key")
}
obj.Signatures = append(obj.Signatures, signature)
}
for i, sig := range parsed.Signatures {
if sig.Protected != nil && len(sig.Protected.bytes()) > 0 {
obj.Signatures[i].protected = &rawHeader{}
err := json.Unmarshal(sig.Protected.bytes(), obj.Signatures[i].protected)
if err != nil {
return nil, err
}
}
// Check that there is not a nonce in the unprotected header
if sig.Header != nil && sig.Header.getNonce() != "" {
return nil, ErrUnprotectedNonce
}
var err error
obj.Signatures[i].Header, err = obj.Signatures[i].mergedHeaders().sanitized()
if err != nil {
return nil, err
}
if obj.Signatures[i].header != nil {
obj.Signatures[i].Unprotected, err = obj.Signatures[i].header.sanitized()
if err != nil {
return nil, err
}
}
if obj.Signatures[i].protected != nil {
obj.Signatures[i].Protected, err = obj.Signatures[i].protected.sanitized()
if err != nil {
return nil, err
}
}
obj.Signatures[i].Signature = sig.Signature.bytes()
// As per RFC 7515 Section 4.1.3, only public keys are allowed to be embedded.
jwk := obj.Signatures[i].Header.JSONWebKey
if jwk != nil && (!jwk.Valid() || !jwk.IsPublic()) {
return nil, errors.New("go-jose/go-jose: invalid embedded jwk, must be public key")
}
// Copy value of sig
original := sig
obj.Signatures[i].header = sig.Header
obj.Signatures[i].original = &original
}
return obj, nil
}
// parseSignedCompact parses a message in compact format.
func parseSignedCompact(input string, payload []byte) (*JSONWebSignature, error) {
parts := strings.Split(input, ".")
if len(parts) != 3 {
return nil, fmt.Errorf("go-jose/go-jose: compact JWS format must have three parts")
}
if parts[1] != "" && payload != nil {
return nil, fmt.Errorf("go-jose/go-jose: payload is not detached")
}
rawProtected, err := base64URLDecode(parts[0])
if err != nil {
return nil, err
}
if payload == nil {
payload, err = base64URLDecode(parts[1])
if err != nil {
return nil, err
}
}
signature, err := base64URLDecode(parts[2])
if err != nil {
return nil, err
}
raw := &rawJSONWebSignature{
Payload: newBuffer(payload),
Protected: newBuffer(rawProtected),
Signature: newBuffer(signature),
}
return raw.sanitized()
}
func (obj JSONWebSignature) compactSerialize(detached bool) (string, error) {
if len(obj.Signatures) != 1 || obj.Signatures[0].header != nil || obj.Signatures[0].protected == nil {
return "", ErrNotSupported
}
serializedProtected := base64.RawURLEncoding.EncodeToString(mustSerializeJSON(obj.Signatures[0].protected))
payload := ""
signature := base64.RawURLEncoding.EncodeToString(obj.Signatures[0].Signature)
if !detached {
payload = base64.RawURLEncoding.EncodeToString(obj.payload)
}
return fmt.Sprintf("%s.%s.%s", serializedProtected, payload, signature), nil
}
// CompactSerialize serializes an object using the compact serialization format.
func (obj JSONWebSignature) CompactSerialize() (string, error) {
return obj.compactSerialize(false)
}
// DetachedCompactSerialize serializes an object using the compact serialization format with detached payload.
func (obj JSONWebSignature) DetachedCompactSerialize() (string, error) {
return obj.compactSerialize(true)
}
// FullSerialize serializes an object using the full JSON serialization format.
func (obj JSONWebSignature) FullSerialize() string {
raw := rawJSONWebSignature{
Payload: newBuffer(obj.payload),
}
if len(obj.Signatures) == 1 {
if obj.Signatures[0].protected != nil {
serializedProtected := mustSerializeJSON(obj.Signatures[0].protected)
raw.Protected = newBuffer(serializedProtected)
}
raw.Header = obj.Signatures[0].header
raw.Signature = newBuffer(obj.Signatures[0].Signature)
} else {
raw.Signatures = make([]rawSignatureInfo, len(obj.Signatures))
for i, signature := range obj.Signatures {
raw.Signatures[i] = rawSignatureInfo{
Header: signature.header,
Signature: newBuffer(signature.Signature),
}
if signature.protected != nil {
raw.Signatures[i].Protected = newBuffer(mustSerializeJSON(signature.protected))
}
}
}
return string(mustSerializeJSON(raw))
}
+334
View File
@@ -0,0 +1,334 @@
/*-
* Copyright 2016 Zbigniew Mandziejewicz
* Copyright 2016 Square, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jwt
import (
"bytes"
"reflect"
"github.com/go-jose/go-jose/v3/json"
"github.com/go-jose/go-jose/v3"
)
// Builder is a utility for making JSON Web Tokens. Calls can be chained, and
// errors are accumulated until the final call to CompactSerialize/FullSerialize.
type Builder interface {
// Claims encodes claims into JWE/JWS form. Multiple calls will merge claims
// into single JSON object. If you are passing private claims, make sure to set
// struct field tags to specify the name for the JSON key to be used when
// serializing.
Claims(i interface{}) Builder
// Token builds a JSONWebToken from provided data.
Token() (*JSONWebToken, error)
// FullSerialize serializes a token using the JWS/JWE JSON Serialization format.
FullSerialize() (string, error)
// CompactSerialize serializes a token using the compact serialization format.
CompactSerialize() (string, error)
}
// NestedBuilder is a utility for making Signed-Then-Encrypted JSON Web Tokens.
// Calls can be chained, and errors are accumulated until final call to
// CompactSerialize/FullSerialize.
type NestedBuilder interface {
// Claims encodes claims into JWE/JWS form. Multiple calls will merge claims
// into single JSON object. If you are passing private claims, make sure to set
// struct field tags to specify the name for the JSON key to be used when
// serializing.
Claims(i interface{}) NestedBuilder
// Token builds a NestedJSONWebToken from provided data.
Token() (*NestedJSONWebToken, error)
// FullSerialize serializes a token using the JSON Serialization format.
FullSerialize() (string, error)
// CompactSerialize serializes a token using the compact serialization format.
CompactSerialize() (string, error)
}
type builder struct {
payload map[string]interface{}
err error
}
type signedBuilder struct {
builder
sig jose.Signer
}
type encryptedBuilder struct {
builder
enc jose.Encrypter
}
type nestedBuilder struct {
builder
sig jose.Signer
enc jose.Encrypter
}
// Signed creates builder for signed tokens.
func Signed(sig jose.Signer) Builder {
return &signedBuilder{
sig: sig,
}
}
// Encrypted creates builder for encrypted tokens.
func Encrypted(enc jose.Encrypter) Builder {
return &encryptedBuilder{
enc: enc,
}
}
// SignedAndEncrypted creates builder for signed-then-encrypted tokens.
// ErrInvalidContentType will be returned if encrypter doesn't have JWT content type.
func SignedAndEncrypted(sig jose.Signer, enc jose.Encrypter) NestedBuilder {
if contentType, _ := enc.Options().ExtraHeaders[jose.HeaderContentType].(jose.ContentType); contentType != "JWT" {
return &nestedBuilder{
builder: builder{
err: ErrInvalidContentType,
},
}
}
return &nestedBuilder{
sig: sig,
enc: enc,
}
}
func (b builder) claims(i interface{}) builder {
if b.err != nil {
return b
}
m, ok := i.(map[string]interface{})
switch {
case ok:
return b.merge(m)
case reflect.Indirect(reflect.ValueOf(i)).Kind() == reflect.Struct:
m, err := normalize(i)
if err != nil {
return builder{
err: err,
}
}
return b.merge(m)
default:
return builder{
err: ErrInvalidClaims,
}
}
}
func normalize(i interface{}) (map[string]interface{}, error) {
m := make(map[string]interface{})
raw, err := json.Marshal(i)
if err != nil {
return nil, err
}
d := json.NewDecoder(bytes.NewReader(raw))
d.SetNumberType(json.UnmarshalJSONNumber)
if err := d.Decode(&m); err != nil {
return nil, err
}
return m, nil
}
func (b *builder) merge(m map[string]interface{}) builder {
p := make(map[string]interface{})
for k, v := range b.payload {
p[k] = v
}
for k, v := range m {
p[k] = v
}
return builder{
payload: p,
}
}
func (b *builder) token(p func(interface{}) ([]byte, error), h []jose.Header) (*JSONWebToken, error) {
return &JSONWebToken{
payload: p,
Headers: h,
}, nil
}
func (b *signedBuilder) Claims(i interface{}) Builder {
return &signedBuilder{
builder: b.builder.claims(i),
sig: b.sig,
}
}
func (b *signedBuilder) Token() (*JSONWebToken, error) {
sig, err := b.sign()
if err != nil {
return nil, err
}
h := make([]jose.Header, len(sig.Signatures))
for i, v := range sig.Signatures {
h[i] = v.Header
}
return b.builder.token(sig.Verify, h)
}
func (b *signedBuilder) CompactSerialize() (string, error) {
sig, err := b.sign()
if err != nil {
return "", err
}
return sig.CompactSerialize()
}
func (b *signedBuilder) FullSerialize() (string, error) {
sig, err := b.sign()
if err != nil {
return "", err
}
return sig.FullSerialize(), nil
}
func (b *signedBuilder) sign() (*jose.JSONWebSignature, error) {
if b.err != nil {
return nil, b.err
}
p, err := json.Marshal(b.payload)
if err != nil {
return nil, err
}
return b.sig.Sign(p)
}
func (b *encryptedBuilder) Claims(i interface{}) Builder {
return &encryptedBuilder{
builder: b.builder.claims(i),
enc: b.enc,
}
}
func (b *encryptedBuilder) CompactSerialize() (string, error) {
enc, err := b.encrypt()
if err != nil {
return "", err
}
return enc.CompactSerialize()
}
func (b *encryptedBuilder) FullSerialize() (string, error) {
enc, err := b.encrypt()
if err != nil {
return "", err
}
return enc.FullSerialize(), nil
}
func (b *encryptedBuilder) Token() (*JSONWebToken, error) {
enc, err := b.encrypt()
if err != nil {
return nil, err
}
return b.builder.token(enc.Decrypt, []jose.Header{enc.Header})
}
func (b *encryptedBuilder) encrypt() (*jose.JSONWebEncryption, error) {
if b.err != nil {
return nil, b.err
}
p, err := json.Marshal(b.payload)
if err != nil {
return nil, err
}
return b.enc.Encrypt(p)
}
func (b *nestedBuilder) Claims(i interface{}) NestedBuilder {
return &nestedBuilder{
builder: b.builder.claims(i),
sig: b.sig,
enc: b.enc,
}
}
func (b *nestedBuilder) Token() (*NestedJSONWebToken, error) {
enc, err := b.signAndEncrypt()
if err != nil {
return nil, err
}
return &NestedJSONWebToken{
enc: enc,
Headers: []jose.Header{enc.Header},
}, nil
}
func (b *nestedBuilder) CompactSerialize() (string, error) {
enc, err := b.signAndEncrypt()
if err != nil {
return "", err
}
return enc.CompactSerialize()
}
func (b *nestedBuilder) FullSerialize() (string, error) {
enc, err := b.signAndEncrypt()
if err != nil {
return "", err
}
return enc.FullSerialize(), nil
}
func (b *nestedBuilder) signAndEncrypt() (*jose.JSONWebEncryption, error) {
if b.err != nil {
return nil, b.err
}
p, err := json.Marshal(b.payload)
if err != nil {
return nil, err
}
sig, err := b.sig.Sign(p)
if err != nil {
return nil, err
}
p2, err := sig.CompactSerialize()
if err != nil {
return nil, err
}
return b.enc.Encrypt([]byte(p2))
}
+130
View File
@@ -0,0 +1,130 @@
/*-
* Copyright 2016 Zbigniew Mandziejewicz
* Copyright 2016 Square, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jwt
import (
"strconv"
"time"
"github.com/go-jose/go-jose/v3/json"
)
// Claims represents public claim values (as specified in RFC 7519).
type Claims struct {
Issuer string `json:"iss,omitempty"`
Subject string `json:"sub,omitempty"`
Audience Audience `json:"aud,omitempty"`
Expiry *NumericDate `json:"exp,omitempty"`
NotBefore *NumericDate `json:"nbf,omitempty"`
IssuedAt *NumericDate `json:"iat,omitempty"`
ID string `json:"jti,omitempty"`
}
// NumericDate represents date and time as the number of seconds since the
// epoch, ignoring leap seconds. Non-integer values can be represented
// in the serialized format, but we round to the nearest second.
// See RFC7519 Section 2: https://tools.ietf.org/html/rfc7519#section-2
type NumericDate int64
// NewNumericDate constructs NumericDate from time.Time value.
func NewNumericDate(t time.Time) *NumericDate {
if t.IsZero() {
return nil
}
// While RFC 7519 technically states that NumericDate values may be
// non-integer values, we don't bother serializing timestamps in
// claims with sub-second accurancy and just round to the nearest
// second instead. Not convined sub-second accuracy is useful here.
out := NumericDate(t.Unix())
return &out
}
// MarshalJSON serializes the given NumericDate into its JSON representation.
func (n NumericDate) MarshalJSON() ([]byte, error) {
return []byte(strconv.FormatInt(int64(n), 10)), nil
}
// UnmarshalJSON reads a date from its JSON representation.
func (n *NumericDate) UnmarshalJSON(b []byte) error {
s := string(b)
f, err := strconv.ParseFloat(s, 64)
if err != nil {
return ErrUnmarshalNumericDate
}
*n = NumericDate(f)
return nil
}
// Time returns time.Time representation of NumericDate.
func (n *NumericDate) Time() time.Time {
if n == nil {
return time.Time{}
}
return time.Unix(int64(*n), 0)
}
// Audience represents the recipients that the token is intended for.
type Audience []string
// UnmarshalJSON reads an audience from its JSON representation.
func (s *Audience) UnmarshalJSON(b []byte) error {
var v interface{}
if err := json.Unmarshal(b, &v); err != nil {
return err
}
switch v := v.(type) {
case string:
*s = []string{v}
case []interface{}:
a := make([]string, len(v))
for i, e := range v {
s, ok := e.(string)
if !ok {
return ErrUnmarshalAudience
}
a[i] = s
}
*s = a
default:
return ErrUnmarshalAudience
}
return nil
}
// MarshalJSON converts audience to json representation.
func (s Audience) MarshalJSON() ([]byte, error) {
if len(s) == 1 {
return json.Marshal(s[0])
}
return json.Marshal([]string(s))
}
//Contains checks whether a given string is included in the Audience
func (s Audience) Contains(v string) bool {
for _, a := range s {
if a == v {
return true
}
}
return false
}
+22
View File
@@ -0,0 +1,22 @@
/*-
* Copyright 2017 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*
Package jwt provides an implementation of the JSON Web Token standard.
*/
package jwt
+53
View File
@@ -0,0 +1,53 @@
/*-
* Copyright 2016 Zbigniew Mandziejewicz
* Copyright 2016 Square, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jwt
import "errors"
// ErrUnmarshalAudience indicates that aud claim could not be unmarshalled.
var ErrUnmarshalAudience = errors.New("go-jose/go-jose/jwt: expected string or array value to unmarshal to Audience")
// ErrUnmarshalNumericDate indicates that JWT NumericDate could not be unmarshalled.
var ErrUnmarshalNumericDate = errors.New("go-jose/go-jose/jwt: expected number value to unmarshal NumericDate")
// ErrInvalidClaims indicates that given claims have invalid type.
var ErrInvalidClaims = errors.New("go-jose/go-jose/jwt: expected claims to be value convertible into JSON object")
// ErrInvalidIssuer indicates invalid iss claim.
var ErrInvalidIssuer = errors.New("go-jose/go-jose/jwt: validation failed, invalid issuer claim (iss)")
// ErrInvalidSubject indicates invalid sub claim.
var ErrInvalidSubject = errors.New("go-jose/go-jose/jwt: validation failed, invalid subject claim (sub)")
// ErrInvalidAudience indicated invalid aud claim.
var ErrInvalidAudience = errors.New("go-jose/go-jose/jwt: validation failed, invalid audience claim (aud)")
// ErrInvalidID indicates invalid jti claim.
var ErrInvalidID = errors.New("go-jose/go-jose/jwt: validation failed, invalid ID claim (jti)")
// ErrNotValidYet indicates that token is used before time indicated in nbf claim.
var ErrNotValidYet = errors.New("go-jose/go-jose/jwt: validation failed, token not valid yet (nbf)")
// ErrExpired indicates that token is used after expiry time indicated in exp claim.
var ErrExpired = errors.New("go-jose/go-jose/jwt: validation failed, token is expired (exp)")
// ErrIssuedInTheFuture indicates that the iat field is in the future.
var ErrIssuedInTheFuture = errors.New("go-jose/go-jose/jwt: validation field, token issued in the future (iat)")
// ErrInvalidContentType indicates that token requires JWT cty header.
var ErrInvalidContentType = errors.New("go-jose/go-jose/jwt: expected content type to be JWT (cty header)")
+133
View File
@@ -0,0 +1,133 @@
/*-
* Copyright 2016 Zbigniew Mandziejewicz
* Copyright 2016 Square, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jwt
import (
"fmt"
"strings"
jose "github.com/go-jose/go-jose/v3"
"github.com/go-jose/go-jose/v3/json"
)
// JSONWebToken represents a JSON Web Token (as specified in RFC7519).
type JSONWebToken struct {
payload func(k interface{}) ([]byte, error)
unverifiedPayload func() []byte
Headers []jose.Header
}
type NestedJSONWebToken struct {
enc *jose.JSONWebEncryption
Headers []jose.Header
}
// Claims deserializes a JSONWebToken into dest using the provided key.
func (t *JSONWebToken) Claims(key interface{}, dest ...interface{}) error {
b, err := t.payload(key)
if err != nil {
return err
}
for _, d := range dest {
if err := json.Unmarshal(b, d); err != nil {
return err
}
}
return nil
}
// UnsafeClaimsWithoutVerification deserializes the claims of a
// JSONWebToken into the dests. For signed JWTs, the claims are not
// verified. This function won't work for encrypted JWTs.
func (t *JSONWebToken) UnsafeClaimsWithoutVerification(dest ...interface{}) error {
if t.unverifiedPayload == nil {
return fmt.Errorf("go-jose/go-jose: Cannot get unverified claims")
}
claims := t.unverifiedPayload()
for _, d := range dest {
if err := json.Unmarshal(claims, d); err != nil {
return err
}
}
return nil
}
func (t *NestedJSONWebToken) Decrypt(decryptionKey interface{}) (*JSONWebToken, error) {
b, err := t.enc.Decrypt(decryptionKey)
if err != nil {
return nil, err
}
sig, err := ParseSigned(string(b))
if err != nil {
return nil, err
}
return sig, nil
}
// ParseSigned parses token from JWS form.
func ParseSigned(s string) (*JSONWebToken, error) {
sig, err := jose.ParseSigned(s)
if err != nil {
return nil, err
}
headers := make([]jose.Header, len(sig.Signatures))
for i, signature := range sig.Signatures {
headers[i] = signature.Header
}
return &JSONWebToken{
payload: sig.Verify,
unverifiedPayload: sig.UnsafePayloadWithoutVerification,
Headers: headers,
}, nil
}
// ParseEncrypted parses token from JWE form.
func ParseEncrypted(s string) (*JSONWebToken, error) {
enc, err := jose.ParseEncrypted(s)
if err != nil {
return nil, err
}
return &JSONWebToken{
payload: enc.Decrypt,
Headers: []jose.Header{enc.Header},
}, nil
}
// ParseSignedAndEncrypted parses signed-then-encrypted token from JWE form.
func ParseSignedAndEncrypted(s string) (*NestedJSONWebToken, error) {
enc, err := jose.ParseEncrypted(s)
if err != nil {
return nil, err
}
contentType, _ := enc.Header.ExtraHeaders[jose.HeaderContentType].(string)
if strings.ToUpper(contentType) != "JWT" {
return nil, ErrInvalidContentType
}
return &NestedJSONWebToken{
enc: enc,
Headers: []jose.Header{enc.Header},
}, nil
}
+120
View File
@@ -0,0 +1,120 @@
/*-
* Copyright 2016 Zbigniew Mandziejewicz
* Copyright 2016 Square, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jwt
import "time"
const (
// DefaultLeeway defines the default leeway for matching NotBefore/Expiry claims.
DefaultLeeway = 1.0 * time.Minute
)
// Expected defines values used for protected claims validation.
// If field has zero value then validation is skipped, with the exception of
// Time, where the zero value means "now." To skip validating them, set the
// corresponding field in the Claims struct to nil.
type Expected struct {
// Issuer matches the "iss" claim exactly.
Issuer string
// Subject matches the "sub" claim exactly.
Subject string
// Audience matches the values in "aud" claim, regardless of their order.
Audience Audience
// ID matches the "jti" claim exactly.
ID string
// Time matches the "exp", "nbf" and "iat" claims with leeway.
Time time.Time
}
// WithTime copies expectations with new time.
func (e Expected) WithTime(t time.Time) Expected {
e.Time = t
return e
}
// Validate checks claims in a token against expected values.
// A default leeway value of one minute is used to compare time values.
//
// The default leeway will cause the token to be deemed valid until one
// minute after the expiration time. If you're a server application that
// wants to give an extra minute to client tokens, use this
// function. If you're a client application wondering if the server
// will accept your token, use ValidateWithLeeway with a leeway <=0,
// otherwise this function might make you think a token is valid when
// it is not.
func (c Claims) Validate(e Expected) error {
return c.ValidateWithLeeway(e, DefaultLeeway)
}
// ValidateWithLeeway checks claims in a token against expected values. A
// custom leeway may be specified for comparing time values. You may pass a
// zero value to check time values with no leeway, but you should note that
// numeric date values are rounded to the nearest second and sub-second
// precision is not supported.
//
// The leeway gives some extra time to the token from the server's
// point of view. That is, if the token is expired, ValidateWithLeeway
// will still accept the token for 'leeway' amount of time. This fails
// if you're using this function to check if a server will accept your
// token, because it will think the token is valid even after it
// expires. So if you're a client validating if the token is valid to
// be submitted to a server, use leeway <=0, if you're a server
// validation a token, use leeway >=0.
func (c Claims) ValidateWithLeeway(e Expected, leeway time.Duration) error {
if e.Issuer != "" && e.Issuer != c.Issuer {
return ErrInvalidIssuer
}
if e.Subject != "" && e.Subject != c.Subject {
return ErrInvalidSubject
}
if e.ID != "" && e.ID != c.ID {
return ErrInvalidID
}
if len(e.Audience) != 0 {
for _, v := range e.Audience {
if !c.Audience.Contains(v) {
return ErrInvalidAudience
}
}
}
// validate using the e.Time, or time.Now if not provided
validationTime := e.Time
if validationTime.IsZero() {
validationTime = time.Now()
}
if c.NotBefore != nil && validationTime.Add(leeway).Before(c.NotBefore.Time()) {
return ErrNotValidYet
}
if c.Expiry != nil && validationTime.Add(-leeway).After(c.Expiry.Time()) {
return ErrExpired
}
// IssuedAt is optional but cannot be in the future. This is not required by the RFC, but
// something is misconfigured if this happens and we should not trust it.
if c.IssuedAt != nil && validationTime.Add(leeway).Before(c.IssuedAt.Time()) {
return ErrIssuedInTheFuture
}
return nil
}
+144
View File
@@ -0,0 +1,144 @@
/*-
* Copyright 2018 Square Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package jose
// OpaqueSigner is an interface that supports signing payloads with opaque
// private key(s). Private key operations performed by implementers may, for
// example, occur in a hardware module. An OpaqueSigner may rotate signing keys
// transparently to the user of this interface.
type OpaqueSigner interface {
// Public returns the public key of the current signing key.
Public() *JSONWebKey
// Algs returns a list of supported signing algorithms.
Algs() []SignatureAlgorithm
// SignPayload signs a payload with the current signing key using the given
// algorithm.
SignPayload(payload []byte, alg SignatureAlgorithm) ([]byte, error)
}
type opaqueSigner struct {
signer OpaqueSigner
}
func newOpaqueSigner(alg SignatureAlgorithm, signer OpaqueSigner) (recipientSigInfo, error) {
var algSupported bool
for _, salg := range signer.Algs() {
if alg == salg {
algSupported = true
break
}
}
if !algSupported {
return recipientSigInfo{}, ErrUnsupportedAlgorithm
}
return recipientSigInfo{
sigAlg: alg,
publicKey: signer.Public,
signer: &opaqueSigner{
signer: signer,
},
}, nil
}
func (o *opaqueSigner) signPayload(payload []byte, alg SignatureAlgorithm) (Signature, error) {
out, err := o.signer.SignPayload(payload, alg)
if err != nil {
return Signature{}, err
}
return Signature{
Signature: out,
protected: &rawHeader{},
}, nil
}
// OpaqueVerifier is an interface that supports verifying payloads with opaque
// public key(s). An OpaqueSigner may rotate signing keys transparently to the
// user of this interface.
type OpaqueVerifier interface {
VerifyPayload(payload []byte, signature []byte, alg SignatureAlgorithm) error
}
type opaqueVerifier struct {
verifier OpaqueVerifier
}
func (o *opaqueVerifier) verifyPayload(payload []byte, signature []byte, alg SignatureAlgorithm) error {
return o.verifier.VerifyPayload(payload, signature, alg)
}
// OpaqueKeyEncrypter is an interface that supports encrypting keys with an opaque key.
type OpaqueKeyEncrypter interface {
// KeyID returns the kid
KeyID() string
// Algs returns a list of supported key encryption algorithms.
Algs() []KeyAlgorithm
// encryptKey encrypts the CEK using the given algorithm.
encryptKey(cek []byte, alg KeyAlgorithm) (recipientInfo, error)
}
type opaqueKeyEncrypter struct {
encrypter OpaqueKeyEncrypter
}
func newOpaqueKeyEncrypter(alg KeyAlgorithm, encrypter OpaqueKeyEncrypter) (recipientKeyInfo, error) {
var algSupported bool
for _, salg := range encrypter.Algs() {
if alg == salg {
algSupported = true
break
}
}
if !algSupported {
return recipientKeyInfo{}, ErrUnsupportedAlgorithm
}
return recipientKeyInfo{
keyID: encrypter.KeyID(),
keyAlg: alg,
keyEncrypter: &opaqueKeyEncrypter{
encrypter: encrypter,
},
}, nil
}
func (oke *opaqueKeyEncrypter) encryptKey(cek []byte, alg KeyAlgorithm) (recipientInfo, error) {
return oke.encrypter.encryptKey(cek, alg)
}
//OpaqueKeyDecrypter is an interface that supports decrypting keys with an opaque key.
type OpaqueKeyDecrypter interface {
DecryptKey(encryptedKey []byte, header Header) ([]byte, error)
}
type opaqueKeyDecrypter struct {
decrypter OpaqueKeyDecrypter
}
func (okd *opaqueKeyDecrypter) decryptKey(headers rawHeader, recipient *recipientInfo, generator keyGenerator) ([]byte, error) {
mergedHeaders := rawHeader{}
mergedHeaders.merge(&headers)
mergedHeaders.merge(recipient.header)
header, err := mergedHeaders.sanitized()
if err != nil {
return nil, err
}
return okd.decrypter.DecryptKey(recipient.encryptedKey, header)
}

Some files were not shown because too many files have changed in this diff Show More