admin/jpillora-chisel
1
0
Fork 0
mirror of https://github.com/jpillora/chisel synced 2026-06-08 15:07:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

ssh working, user auth working, addr whitelisting (mostly) working

Browse Source
This commit is contained in:
Jaime Pillora
2015-03-19 02:41:17 +11:00
parent 68eb79a634
commit 1871137e32
8 changed files with 154 additions and 258 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+58 -35
View File
@@ -8,7 +8,7 @@ Chisel is an HTTP client and server which acts as a TCP proxy, written in Go (Go
**Binaries**
See [Releases](https://github.com/jpillora/chisel/releases)
See [Releases](https://github.com/jpillora/chisel/releases/latest)
**Source**
@@ -20,7 +20,8 @@ $ go get -v github.com/jpillora/chisel
* Easy to use
* [Performant](#performance)*
* [Encrypted connections](https://github.com/jpillora/conncrypt) with `key` derived (PBKDF2) symmetric key[*](#security)
* [Encrypted connections](#security) using `crypto/ssh`
* [Authenticated connections](#authentication) using a users config file
* Client auto-reconnects with [exponential backoff](https://github.com/jpillora/backoff)
* Client can create multiple tunnel endpoints over one TCP connection
* Server optionally doubles as a [reverse proxy](http://golang.org/pkg/net/http/httputil/#NewSingleHostReverseProxy)
@@ -30,22 +31,19 @@ $ go get -v github.com/jpillora/chisel
A [demo app](https://chisel-demo.herokuapp.com) on Heroku is running this `chisel server`:
``` sh
$ chisel server --key foobar --port $PORT --proxy http://example.com
# listens on $PORT, requires password 'foobar', proxy web requests to 'http://example.com'
$ chisel server --port $PORT --proxy http://example.com
# listens on $PORT, proxy web requests to 'http://example.com'
```
This demo app is also running a [simple file server](https://www.npmjs.com/package/serve) on `:3000`, which is normally inaccessible due to Heroku's firewall. However, if we tunnel in with:
``` sh
$ chisel client --key foobar https://chisel-demo.herokuapp.com 3000
# connects to 'https://chisel-demo.herokuapp.com', using password 'foobar',
$ chisel client https://chisel-demo.herokuapp.com 3000
# connects to 'https://chisel-demo.herokuapp.com',
# tunnels your localhost:3000 to the server's localhost:3000
```
and then visit [localhost:3000](http://localhost:3000/), we should
see a directory listing of the demo app's root. Also, if we visit
the [demo app](https://chisel-demo.herokuapp.com) in the browser we should hit the server's
default proxy and see a copy of [example.com](http://example.com).
and then visit [localhost:3000](http://localhost:3000/), we should see a directory listing of the demo app's root. Also, if we visit the [demo app](https://chisel-demo.herokuapp.com) in the browser we should hit the server's default proxy and see a copy of [example.com](http://example.com).
### Usage
@@ -54,7 +52,7 @@ default proxy and see a copy of [example.com](http://example.com).
Usage: chisel [command] [--help]
Version: X.X.X
Version: 0.0.0-src
Commands:
server - runs chisel in server mode
@@ -66,6 +64,8 @@ default proxy and see a copy of [example.com](http://example.com).
```
</tmpl>
`chisel server --help`
<tmpl,code: chisel server --help>
```
@@ -78,13 +78,22 @@ default proxy and see a copy of [example.com](http://example.com).
--port, Defines the HTTP listening port (defaults to 8080).
--key, An optional string to seed the generation of a ECC public
and private key pair. All commications will be secured using this
key pair. Share the resulting fingerprint with clients to prevent
man-in-the-middle attacks.
--authfile, An optional path to a users.json file. This file should
be an object with users defined like:
"<user:pass>": ["<addr-regex>","<addr-regex>"]
when <user> connects, their <pass> will be verified and then
each of the remote addresses will be compared against the list
of address regular expressions for a match. Addresses will
always come in the form "<host/ip>:<port>".
--proxy, Specifies the default proxy target to use when chisel
receives a normal HTTP request.
--key, Enables AES256 encryption and specify the string to
use to derive the key (derivation is performed using PBKDF2
with 2048 iterations of SHA256).
-v, Enable verbose logging
--help, This help text
@@ -95,6 +104,8 @@ default proxy and see a copy of [example.com](http://example.com).
```
</tmpl>
`chisel client --help`
<tmpl,code: chisel client --help>
```
@@ -121,9 +132,14 @@ default proxy and see a copy of [example.com](http://example.com).
Options:
--key, Enables AES256 encryption and specify the string to
use to derive the key (derivation is performed using PBKDF2
with 2048 iterations of SHA256).
--fingerprint, An optional fingerprint (server authentication)
string to compare against the server's public key. You may provide
just a prefix of the key or the entire string. Fingerprint
mismatches will close the connection.
--auth, An optional username and password (client authentication)
in the form: "<user>:<pass>". These credentials are compared to
the credentials inside the server's --authfile.
-v, Enable verbose logging
@@ -139,13 +155,17 @@ See also [programmatic usage](https://github.com/jpillora/chisel/wiki/Programmat
### Security
**Beware** The `key` option derives the keys and initialization vectors and is currently susceptible to a sustained targeted attack, this risk will be lessened when the switch SSH is complete.
Encryption is enabled by default, when you start up a chisel server, it will generate an in-memory ECC public/private key pair. The public key fingerprint will be displayed as the server starts. Instead of always generating a random key, the server may optionally specify a key seed, using the `--key`, which will be used to seed the key generation. When clients connect, they will also display the server's public key fingerprint. The client can force a particular fingerprint using the `--fingerprint` option. See the `--help` above for more information.
It's recommended to use TLS to secure your traffic, which can only be done by hosting your chisel server behind a TLS terminating proxy (like Heroku's router). In the future, the server will allow your to pass in TLS credentials and make use of Go's TLS (HTTPS) server.
### Authentication
Using the `--authfile` option, the server may optionally provide a `user.json` configuration file to create a list of accepted users. The client then authenticates using the `--auth` option. See [users.json](example/users.json) for an example authentication configuration file. See the `--help` above for more information.
Internally, this is done using the *Password* authentication method provided by SSH. Learn more about `crypto/ssh` here http://blog.gopheracademy.com/go-and-ssh/.
### Performance
With [crowbar](https://github.com/q3k/crowbar), a connection is tunnelled by repeatedly querying the server with updates. This results in a large amount of HTTP and TCP connection overhead. Chisel overcomes this using WebSockets combined with [Yamux](https://github.com/hashicorp/yamux) to create hundreds of SDPY/HTTP2 like logical connections, resulting in **one** TCP connection per client.
With [crowbar](https://github.com/q3k/crowbar), a connection is tunnelled by repeatedly querying the server with updates. This results in a large amount of HTTP and TCP connection overhead. Chisel overcomes this using WebSockets combined with [crypto/ssh](https://golang.org/x/crypto/ssh) to create hundreds of logical connections, resulting in **one** TCP connection per client.
In this simple benchmark, we have:
@@ -178,18 +198,18 @@ Note, we're using an in-memory "file" server on localhost for these tests
`chisel`
```
:2001 => 1 bytes in 1.334661ms
:2001 => 10 bytes in 807.797µs
:2001 => 100 bytes in 763.728µs
:2001 => 1000 bytes in 1.029811ms
:2001 => 10000 bytes in 840.247µs
:2001 => 100000 bytes in 1.647748ms
:2001 => 1000000 bytes in 3.495904ms
:2001 => 10000000 bytes in 22.298904ms
:2001 => 100000000 bytes in 255.410448ms
:2001 => 1 bytes in 1.190288ms
:2001 => 10 bytes in 1.17237ms
:2001 => 100 bytes in 821.369µs
:2001 => 1000 bytes in 1.029366ms
:2001 => 10000 bytes in 1.281065ms
:2001 => 100000 bytes in 2.14094ms
:2001 => 1000000 bytes in 9.538984ms
:2001 => 10000000 bytes in 86.500426ms
:2001 => 100000000 bytes in 814.630443ms
```
~100MB in **a quarter of a second**
~100MB in **0.8 seconds**
`crowbar`
@@ -227,13 +247,16 @@ See more [test/](test/)
* `github.com/jpillora/chisel/server` contains the server package
* `github.com/jpillora/chisel/client` contains the client package
### Changelog
* `1.0.0` - Init
* `1.1.0` - Swapped out simple symmetric encryption for ECC SSH
### Todo
* Users file with white-listed remotes
* Pass in TLS server configuration
* Better, faster tests
* Expose a stats page for proxy throughput
* Configurable connection retry times
* Treat forwarder stdin/stdout as a socket
* Treat client stdin/stdout as a socket
#### MIT License
client/client.go
+21 -21
View File
@@ -23,12 +23,11 @@ type Client struct {
sshConn ssh.Conn
fingerprint string
server string
keyPrefix string
running bool
runningc chan error
}
func NewClient(keyPrefix, auth, server string, remotes ...string) (*Client, error) {
func NewClient(fingerprint, auth, server string, remotes ...string) (*Client, error) {
//apply default scheme
if !strings.HasPrefix(server, "http") {
@@ -62,23 +61,21 @@ func NewClient(keyPrefix, auth, server string, remotes ...string) (*Client, erro
}
c := &Client{
Logger: chshare.NewLogger("client"),
config: config,
server: u.String(),
keyPrefix: keyPrefix,
running: true,
runningc: make(chan error, 1),
}
c.sshConfig = &ssh.ClientConfig{
ClientVersion: chshare.ProtocolVersion + "-client",
HostKeyCallback: c.verifyServer,
Logger: chshare.NewLogger("client"),
config: config,
server: u.String(),
fingerprint: fingerprint,
running: true,
runningc: make(chan error, 1),
}
user, pass := chshare.ParseAuth(auth)
if user != "" {
c.sshConfig.User = user
c.sshConfig.Auth = []ssh.AuthMethod{ssh.Password(pass)}
c.sshConfig = &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{ssh.Password(pass)},
ClientVersion: chshare.ProtocolVersion + "-client",
HostKeyCallback: c.verifyServer,
}
return c, nil
@@ -92,10 +89,11 @@ func (c *Client) Run() error {
func (c *Client) verifyServer(hostname string, remote net.Addr, key ssh.PublicKey) error {
f := chshare.FingerprintKey(key)
if c.keyPrefix != "" && !strings.HasPrefix(f, c.keyPrefix) {
if c.fingerprint != "" && !strings.HasPrefix(f, c.fingerprint) {
return fmt.Errorf("Invalid fingerprint (Got %s)", f)
}
c.fingerprint = f
//overwrite with complete fingerprint
c.Infof("Fingerprint %s", f)
return nil
}
@@ -140,15 +138,17 @@ func (c *Client) start() {
if err != nil {
if strings.Contains(err.Error(), "unable to authenticate") {
c.Infof("Authentication failed")
c.Debugf(err.Error())
} else {
c.Infof(err.Error())
}
break
}
conf, _ := chshare.EncodeConfig(c.config)
c.Debugf("Sending configurating")
_, conerr, err := sshConn.SendRequest("config", true, conf)
if err != nil {
c.Infof("Config verification failed", c.fingerprint)
c.Infof("Config verification failed")
break
}
if len(conerr) > 0 {
@@ -156,13 +156,13 @@ func (c *Client) start() {
break
}
c.Infof("Connected (%s)", c.fingerprint)
c.Infof("Connected")
//connected
b.Reset()
c.sshConn = sshConn
go ssh.DiscardRequests(reqs)
go chshare.RejectStreams(chans)
go chshare.RejectStreams(chans) //TODO allow client to ConnectStreams
err = sshConn.Wait()
//disconnected
c.sshConn = nil
example/users.json
+6 -3
View File
@@ -1,9 +1,12 @@
{
"root:toor": [
""
],
"foo:bar": [
"0.0.0.0:3000"
"^0.0.0.0:3000$"
],
"ping:pong": [
"0.0.0.0:[45]000",
"example.com:80"
"^0.0.0.0:[45]000$",
"^example.com:80$"
]
}
main.go
+8 -7
View File
@@ -163,20 +163,21 @@ var clientHelp = `
Options:
--key, An optional fingerprint (server authentication) string to
compare against the server's public key. You may provide just a
prefix of the key or the entire string. Fingerprint mismatches
will disallow the connection.
--fingerprint, An optional fingerprint (server authentication)
string to compare against the server's public key. You may provide
just a prefix of the key or the entire string. Fingerprint
mismatches will close the connection.
--auth, An optional username and password (client authentication)
in the form: "<user>:<pass>".
in the form: "<user>:<pass>". These credentials are compared to
the credentials inside the server's --authfile.
` + commonHelp
func client(args []string) {
flags := flag.NewFlagSet("client", flag.ContinueOnError)
key := flags.String("key", "", "")
fingerprint := flags.String("fingerprint", "", "")
auth := flags.String("auth", "", "")
verbose := flags.Bool("v", false, "")
flags.Usage = func() {
@@ -193,7 +194,7 @@ func client(args []string) {
server := args[0]
remotes := args[1:]
c, err := chclient.NewClient(*key, *auth, server, remotes...)
c, err := chclient.NewClient(*fingerprint, *auth, server, remotes...)
if err != nil {
log.Fatal(err)
}
server/server.go
+15 -18
View File
@@ -125,13 +125,15 @@ func (s *Server) handleHTTP(w http.ResponseWriter, r *http.Request) {
//
func (s *Server) authUser(c ssh.ConnMetadata, pass []byte) (*ssh.Permissions, error) {
// no auth
// no auth - allow all
if len(s.Users) == 0 {
return nil, nil
}
// authenticate user
u, ok := s.Users[c.User()]
n := c.User()
u, ok := s.Users[n]
if !ok || u.Pass != string(pass) {
s.Debugf("Login failed: %s", n)
return nil, errors.New("Invalid auth")
}
//insert session
@@ -148,42 +150,41 @@ func (s *Server) handleWS(ws *websocket.Conn) {
}
//load user
sid := string(sshConn.SessionID())
var user *chshare.User
if len(s.Users) > 0 {
sid := string(sshConn.SessionID())
user = s.sessions[sid]
defer delete(s.sessions, sid)
}
//verify configuration
s.Debugf("Verifying configuration")
r := <-reqs
reply := func(err error) {
r.Reply(err == nil, []byte(err.Error()))
if err != nil {
sshConn.Close()
}
failed := func(err error) {
r.Reply(false, []byte(err.Error()))
}
if r.Type != "config" {
reply(s.Errorf("expecting config request"))
failed(s.Errorf("expecting config request"))
return
}
c, err := chshare.DecodeConfig(r.Payload)
if err != nil {
reply(s.Errorf("invalid config"))
failed(s.Errorf("invalid config"))
return
}
//if user is provided, ensure they have
//access to the desired remote
//access to the desired remotes
if user != nil {
for _, r := range c.Remotes {
addr := r.RemoteHost + ":" + r.RemotePort
if !user.HasAccess(addr) {
reply(s.Errorf("access to '%s' denied", addr))
failed(s.Errorf("access to '%s' denied", addr))
return
}
}
}
//success!
r.Reply(true, nil)
//prepare connection logger
s.wsCount++
@@ -195,8 +196,4 @@ func (s *Server) handleWS(ws *websocket.Conn) {
go chshare.ConnectStreams(l, chans)
sshConn.Wait()
l.Debugf("Close")
if user != nil {
delete(s.sessions, sid)
}
}
share/determ_rand.go
+4 -4
View File
@@ -1,14 +1,14 @@
package chshare
// overview: only half the result is used as the output
// next -> sha512(next) -> [output|next] ->
// overview: half the result is used as the output
// [a|...] -> sha512(a) -> [b|output] -> sha512(b)
import (
"crypto/sha512"
"io"
)
const DetermRandIter = 1024
const DetermRandIter = 2048
func NewDetermRand(seed []byte) io.Reader {
var out []byte
@@ -38,7 +38,7 @@ func (d *DetermRand) Read(b []byte) (int, error) {
return n, nil
}
func hash(input []byte) ([]byte, []byte) {
func hash(input []byte) (next []byte, output []byte) {
nextout := sha512.Sum512(input)
return nextout[:sha512.Size/2], nextout[sha512.Size/2:]
}
test/chisel_test.go
-142
View File
@@ -1,142 +0,0 @@
//chisel tests
//===================
// (direct)
// .--------------->----------------.
// / chisel chisel \
// request--->client:2001--->server:2002---->fileserver:3000
// \ /
// '--> crowbar:4001--->crowbar:4002'
// client server
//
// benchmarks don't use testing.B, instead use
// go test -test.run=Bench
//
// tests use
// go test -test.run=Request
//
// crowbar and chisel binaries should be in your PATH
package test
import (
"fmt"
"io/ioutil"
"log"
"os"
"os/exec"
"path"
"testing"
"time"
)
const (
B = 1
KB = 1024 * B
MB = 1024 * KB
GB = 1024 * MB
)
//test
func TestRequestChisel(t *testing.T) {
testTunnel("2001", 500, t)
testTunnel("2001", 50000, t)
}
//benchmark
func TestBenchDirect(t *testing.T) {
benchSizes("3000", t)
}
func TestBenchChisel(t *testing.T) {
benchSizes("2001", t)
}
// func TestBenchrowbar(t *testing.T) {
// benchSizes("4001", t)
// }
func benchSizes(port string, t *testing.T) {
for size := 1; size < 100*MB; size *= 10 {
testTunnel(port, size, t)
}
}
func testTunnel(port string, size int, t *testing.T) {
t0 := time.Now()
resp, err := requestFile(port, size)
if err != nil {
t.Fatal(err)
}
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
t.Fatal(err)
}
t1 := time.Now()
fmt.Printf(":%s => %d bytes in %s\n", port, size, t1.Sub(t0))
if len(b) != size {
t.Fatalf("%d bytes expected, got %d", size, len(b))
}
}
//============================
//global setup
func TestMain(m *testing.M) {
fs := makeFileServer()
go func() {
log.Fatal(fs.Wait())
}()
dir, _ := os.Getwd()
cd := exec.Command("crowbard",
`-listen`, "0.0.0.0:4002",
`-userfile`, path.Join(dir, "userfile"))
if err := cd.Start(); err != nil {
log.Fatal(err)
}
go func() {
log.Fatalf("crowbard: %s", cd.Wait())
}()
time.Sleep(100 * time.Millisecond)
cf := exec.Command("crowbar-forward",
"-local=0.0.0.0:4001",
"-server=http://127.0.0.1:4002",
"-remote=127.0.0.1:3000",
"-username", "foo",
"-password", "bar")
if err := cf.Start(); err != nil {
log.Fatal(err)
}
time.Sleep(100 * time.Millisecond)
hd := exec.Command("chiseld", "--port", "2002", "--auth", "foobar")
// hd.Stdout = os.Stdout
if err := hd.Start(); err != nil {
log.Fatal(err)
}
hf := exec.Command("chisel-forward",
"--auth", "foobar",
"127.0.0.1:2002",
"2001:3000")
if err := hf.Start(); err != nil {
log.Fatal(err)
}
time.Sleep(300 * time.Millisecond)
fmt.Println("Running!")
code := m.Run()
fmt.Println("Done")
cd.Process.Kill()
cf.Process.Kill()
hd.Process.Kill()
hf.Process.Kill()
fs.Close()
os.Exit(code)
}
test/main.go
+42 -28
View File
@@ -29,6 +29,8 @@ import (
"time"
)
const ENABLE_CROWBAR = false
const (
B = 1
KB = 1024 * B
@@ -62,7 +64,9 @@ func test() {
func bench() {
benchSizes("3000")
benchSizes("2001")
benchSizes("4001")
if ENABLE_CROWBAR {
benchSizes("4001")
}
}
func benchSizes(port string) {
@@ -77,6 +81,9 @@ func testTunnel(port string, size int) {
if err != nil {
fatal(err)
}
if resp.StatusCode != 200 {
fatal(err)
}
b, err := ioutil.ReadAll(resp.Body)
if err != nil {
fatal(err)
@@ -135,46 +142,57 @@ func main() {
fatal(fs.Wait())
}()
dir, _ := os.Getwd()
cd := exec.Command("crowbard",
`-listen`, "0.0.0.0:4002",
`-userfile`, path.Join(dir, "userfile"))
if err := cd.Start(); err != nil {
fatal(err)
}
go func() {
fatalf("crowbard: %v", cd.Wait())
}()
if ENABLE_CROWBAR {
dir, _ := os.Getwd()
cd := exec.Command("crowbard",
`-listen`, "0.0.0.0:4002",
`-userfile`, path.Join(dir, "userfile"))
if err := cd.Start(); err != nil {
fatal(err)
}
go func() {
fatalf("crowbard: %v", cd.Wait())
}()
defer cd.Process.Kill()
time.Sleep(100 * time.Millisecond)
time.Sleep(100 * time.Millisecond)
cf := exec.Command("crowbar-forward",
"-local=0.0.0.0:4001",
"-server=http://127.0.0.1:4002",
"-remote=127.0.0.1:3000",
"-username", "foo",
"-password", "bar")
if err := cf.Start(); err != nil {
fatal(err)
cf := exec.Command("crowbar-forward",
"-local=0.0.0.0:4001",
"-server=http://127.0.0.1:4002",
"-remote=127.0.0.1:3000",
"-username", "foo",
"-password", "bar")
if err := cf.Start(); err != nil {
fatal(err)
}
defer cf.Process.Kill()
}
time.Sleep(100 * time.Millisecond)
hd := exec.Command("chisel", "server",
// "--key", "foobar",
// "-v",
"--key", "foobar",
"--port", "2002")
// hd.Stdout = os.Stdout
hd.Stdout = os.Stdout
if err := hd.Start(); err != nil {
fatal(err)
}
defer hd.Process.Kill()
time.Sleep(100 * time.Millisecond)
hf := exec.Command("chisel", "client",
// "--key", "foobar",
// "-v",
"--fingerprint", "ed:f2:cf:3c:56",
"127.0.0.1:2002",
"2001:3000")
// hf.Stdout = os.Stdout
hf.Stdout = os.Stdout
if err := hf.Start(); err != nil {
fatal(err)
}
defer hf.Process.Kill()
time.Sleep(100 * time.Millisecond)
@@ -185,9 +203,5 @@ func main() {
}()
run()
cd.Process.Kill()
cf.Process.Kill()
hd.Process.Kill()
hf.Process.Kill()
fs.Close()
}