mirror of
https://github.com/activecm/rita
synced 2026-06-08 13:02:45 +00:00
b923c39e2d
* Update on config structure, functionality, and tests Co-Authored-By: Naomi Kramer <naomiagoddard@gmail.com> * Extend subnet type to read/write from db, update tests Co-Authored-By: Liza Tsibur <liza@activecountermeasures.com> * updated read file config test and subnet tests * fixed config and util tests, updated subnet related functions Co-Authored-By: Naomi Kramer <naomiagoddard@gmail.com> * Remove error return from GetDefaultConfig Co-Authored-By: Liza Tsibur <liza@activecountermeasures.com> * added json tags to database struct * Updated beacon weights validation for config * updates to score thresholds validation tags * changes to config subnet validation and testing * Update subnet.go * Write missing host entries to http to populate http_proto * Updating some fields to uint64 * WIP update some field types * Update zeek count types and fix tests * Add clickhouse credentials * Misc fixes * Update pointer * Add ability to mark datasets as sample datasets * fix column name * Fix datasets exiting import if hour is empty * Fix zeek count parsing from TSV files * Remove storing dns conns in arrays, Fix historical first seen dns lag * Remove unused columns * Update config.hjson * Update config.hjson * updated impact category score functions to use float64 Co-Authored-By: Naomi Kramer <naomiagoddard@gmail.com> * Update subnet.go * Store import version in imports table * Fix duplicated SNI/IP long connections * Update subnet_test.go * Cleanup output * Rolling files updates (#39) * Limit number of days to import for rolling datasets * Fix breaking imports when import was interrupted * Remove debug output --------- Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com> * Omit parts of env from output * Set max for threat intel datasize * Remove SELinux neutering for QA * Add network size column * Fix http_proto for missing host, update tests for missing host fixes * Add online feeds to default config * Update sshprep (#45) * Update sshprep Co-Authored-By: William Stearns <3538265+william-stearns@users.noreply.github.com> * Update sshprep Add Bradley's suggestion of using head -1 to limit to a single address. --------- Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com> Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com> * Installer Behavior Tweaks (#41) * Add --yes flag to add-apt-repository command * Add missing sudo flags, make sure we're using the SUDO variable instead * Add ability to perform zone transfers (#48) * Store zone transfer records Co-Authored-By: moth <25512187+0x6d6f7468@users.noreply.github.com> * Update config * Add tests * Tests, connectivity test * Update tests --------- Co-authored-by: moth <25512187+0x6d6f7468@users.noreply.github.com> * Support RedHat/RHEL as a valid target (#47) * Update sshprep Co-Authored-By: William Stearns <3538265+william-stearns@users.noreply.github.com> * Supporrt RedHat/RHEL as a valid target --------- Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com> Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com> Co-authored-by: moth <moth@blackhillsinfosec.com> * Fix tests (#49) * Fix tests * Update WalkFiles to use UTC * fixed issue with rolling datasets over 24hours old not getting historical first seen timestamp set (#52) * Change values from float32 to float64 (#50) * Switch float32 to float64 * Update threat category calculation to match CalculateBucketedScore (#51) --------- Co-authored-by: Liza Tsibur <liza@activecountermeasures.com> * Bump max query execution time default value * Use string instead of error for ZoneTransferConnectivityErrors struct fields (#61) * Upgrade Golang to version 1.24 (#59) (#60) * Replace get_url with shell and curl (#58) * Update sshprep Co-Authored-By: William Stearns <3538265+william-stearns@users.noreply.github.com> * Replace get_url with shell and curl * Use get_url by default, fall back to curl if it fails --------- Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com> Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com> Co-authored-by: moth <moth@blackhillsinfosec.com> * add automated log transfer, AC-Hunter issue 135 (#62) * Update sshprep Co-Authored-By: William Stearns <3538265+william-stearns@users.noreply.github.com> * add automated log transfer, PR135 * cron requires non-executable permission * Specify suggested YAML plugin and config in VSCode workspace * Linting and light cleanup * Update generate_installer.sh Download zeek_log_transport.sh to send to the sensor. * Create cron file if remote zeek installation * Only run zeek log import steps for remote sensor installations --------- Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com> Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com> Co-authored-by: moth <moth@blackhillsinfosec.com> * Temporarily disable RITA/Zeek log transport until installer is modular (#66) * Uniform -y flag usage for repo management/package installation; Uniform SUDO variable usage (#68) * Resolve Installer Side Effects and Formalize RHEL Support (#73) * Add missing necessary wildcards for RHEL versions * Remove Ansible task replacing python3-requests to avoid RHEL distro installation side effects * Update supported distros in README * Update scoring defaults * Resolve Ansible Reboot Errors (#75) * Clean up conditionals; Fix reboot step for Ubuntu * Suppress erroneous error output on RPM systems, ignore errors on reboot necessity checks * Ignore missing host rows for openhttp (#76) * Fix integration tests due to prevalence (#77) --------- Co-authored-by: Liza Tsibur <liza@activecountermeasures.com> Co-authored-by: moth <moth@blackhillsinfosec.com> Co-authored-by: William Stearns <william.l.stearns@gmail.com> Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com> Co-authored-by: moth <25512187+0x6d6f7468@users.noreply.github.com>
474 lines
12 KiB
Go
474 lines
12 KiB
Go
package viewer
|
|
|
|
import (
|
|
"fmt"
|
|
"net/netip"
|
|
"regexp"
|
|
"slices"
|
|
"strconv"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/activecm/rita/v5/config"
|
|
"github.com/activecm/rita/v5/util"
|
|
|
|
"github.com/charmbracelet/bubbles/textinput"
|
|
tea "github.com/charmbracelet/bubbletea"
|
|
"github.com/charmbracelet/lipgloss"
|
|
)
|
|
|
|
var (
|
|
operatorRegex = regexp.MustCompile(`^(?P<operator>[><]=?)?(?P<value>(\d|[A-Za-z.])+)$`)
|
|
|
|
allowedSortColumns = []string{"severity", "beacon", "duration", "subdomains"}
|
|
|
|
numericalColumns = []string{"count", "beacon", "subdomains"}
|
|
|
|
// any columns in percentage columns must also be listed in numerical columns
|
|
percentageColumns = []string{"beacon"}
|
|
|
|
timeColumns = []string{"duration"}
|
|
|
|
stringColumns = []string{"src", "dst", "severity", "sort", "threat_intel"}
|
|
)
|
|
|
|
var searchStyle = lipgloss.NewStyle().MarginTop(3)
|
|
|
|
type OperatorFilter struct {
|
|
Operator string
|
|
Value string
|
|
}
|
|
type Filter struct {
|
|
Src string
|
|
Dst string
|
|
Fqdn string
|
|
Severity []OperatorFilter
|
|
Count OperatorFilter
|
|
Beacon OperatorFilter
|
|
Duration OperatorFilter
|
|
Subdomains OperatorFilter
|
|
ThreatIntel string
|
|
SortSeverity string
|
|
SortBeacon string
|
|
SortDuration string
|
|
SortSubdomains string
|
|
// For testing
|
|
LastSeen time.Time
|
|
SortLastSeen string
|
|
}
|
|
|
|
type searchModel struct {
|
|
initialValue string
|
|
TextInput textinput.Model
|
|
width int
|
|
searchErr string
|
|
// filters Filter
|
|
}
|
|
|
|
func NewSearchModel(initialValue string, width int) searchModel {
|
|
ti := textinput.New()
|
|
ti.Placeholder = ""
|
|
ti.Focus()
|
|
ti.PromptStyle = ti.PromptStyle.Foreground(mauve)
|
|
ti.TextStyle = ti.TextStyle.Faint(true)
|
|
ti.Blur()
|
|
ti.SetValue(initialValue)
|
|
ti.CursorStart()
|
|
|
|
return searchModel{
|
|
TextInput: ti,
|
|
initialValue: initialValue,
|
|
width: width,
|
|
}
|
|
}
|
|
|
|
func (m *searchModel) Init() tea.Cmd {
|
|
return textinput.Blink
|
|
}
|
|
|
|
func (m *searchModel) Update(msg tea.Msg) (*searchModel, tea.Cmd) {
|
|
var cmd tea.Cmd
|
|
m.TextInput, cmd = m.TextInput.Update(msg)
|
|
return m, cmd
|
|
}
|
|
|
|
func (m *searchModel) View() string {
|
|
helpStyle := lipgloss.NewStyle().Foreground(overlay0)
|
|
subduedHelpStyle := lipgloss.NewStyle().Foreground(surface0)
|
|
var label string
|
|
if m.searchErr != "" {
|
|
m.TextInput.Prompt = ""
|
|
label = lipgloss.NewStyle().Foreground(red).Render(m.searchErr)
|
|
} else {
|
|
if m.TextInput.Focused() {
|
|
m.TextInput.Prompt = ""
|
|
label = lipgloss.JoinHorizontal(lipgloss.Left,
|
|
helpStyle.Render("enter"),
|
|
" ",
|
|
subduedHelpStyle.Render("submit"),
|
|
" ",
|
|
subduedHelpStyle.Render(bullet),
|
|
" ",
|
|
helpStyle.Render("esc"),
|
|
" ",
|
|
subduedHelpStyle.Render("cancel search"),
|
|
" ",
|
|
subduedHelpStyle.Render(bullet),
|
|
" ",
|
|
helpStyle.Render("ctrl+x"),
|
|
" ",
|
|
subduedHelpStyle.Render("clear"),
|
|
" ",
|
|
subduedHelpStyle.Render(bullet),
|
|
" ",
|
|
helpStyle.Render("?"),
|
|
" ",
|
|
subduedHelpStyle.Render("toggle help"),
|
|
)
|
|
|
|
} else {
|
|
label = helpStyle.Render("press / to begin search")
|
|
if m.TextInput.Value() == "" {
|
|
m.TextInput.Prompt = "Search: "
|
|
} else {
|
|
|
|
label = lipgloss.JoinHorizontal(lipgloss.Left,
|
|
label,
|
|
" ",
|
|
subduedHelpStyle.Render("edit"),
|
|
" ",
|
|
subduedHelpStyle.Render(bullet),
|
|
" ",
|
|
helpStyle.Render("ctrl+x"),
|
|
" ",
|
|
subduedHelpStyle.Render("clear filter"),
|
|
)
|
|
m.TextInput.Prompt = ""
|
|
}
|
|
}
|
|
}
|
|
help := lipgloss.NewStyle().
|
|
MarginLeft(1).
|
|
Foreground(helpTextColor).
|
|
Render(label)
|
|
input := lipgloss.NewStyle().
|
|
Width(m.width).
|
|
Border(lipgloss.RoundedBorder()).
|
|
BorderForeground(overlay0).
|
|
Render(m.TextInput.View())
|
|
|
|
return searchStyle.Render(lipgloss.JoinVertical(lipgloss.Top, help, input))
|
|
}
|
|
|
|
func (m *searchModel) Focus() {
|
|
m.TextInput.TextStyle = m.TextInput.TextStyle.Faint(false)
|
|
m.TextInput.CursorEnd()
|
|
m.TextInput.Focus()
|
|
}
|
|
|
|
func (m *searchModel) Blur() {
|
|
m.TextInput.TextStyle = m.TextInput.TextStyle.Faint(true)
|
|
m.TextInput.Blur()
|
|
}
|
|
|
|
func (m *searchModel) HasError() bool {
|
|
return m.searchErr != ""
|
|
}
|
|
|
|
func (m *searchModel) SetValue(val string) {
|
|
m.TextInput.SetValue(val)
|
|
}
|
|
|
|
func (m *searchModel) Value() string {
|
|
return m.TextInput.Value()
|
|
}
|
|
|
|
func (m *searchModel) ValidateSearchInput() {
|
|
// reset search error and check for commas as user is typing
|
|
switch {
|
|
case strings.Contains(m.Value(), ","):
|
|
m.searchErr = "commas are not supported"
|
|
default:
|
|
m.searchErr = ""
|
|
|
|
}
|
|
|
|
// split input on space to make sure user was finished entering a search field before validating
|
|
split := strings.Split(m.Value(), " ")
|
|
if len(split) > 1 {
|
|
// parse search input for errors
|
|
if _, err := ParseSearchInput(m.Value()); err != "" {
|
|
m.searchErr = err
|
|
}
|
|
}
|
|
}
|
|
|
|
func (m *searchModel) Filter() *Filter {
|
|
filter, err := ParseSearchInput(m.TextInput.Value())
|
|
if err != "" {
|
|
m.searchErr = err
|
|
}
|
|
return filter
|
|
}
|
|
|
|
// ParseSearchInput parses the search input and returns a filter struct
|
|
func ParseSearchInput(input string) (*Filter, string) {
|
|
// create a new filter struct
|
|
var criteria Filter
|
|
|
|
// return an empty filter if input is empty
|
|
if input == "" {
|
|
return nil, ""
|
|
}
|
|
|
|
// check for commas in the input
|
|
if strings.Contains(input, ",") {
|
|
return nil, "commas are not supported"
|
|
}
|
|
|
|
// split input into field-value pairs
|
|
pairs := strings.Fields(input)
|
|
|
|
for _, input := range pairs {
|
|
// skip validation if there was a space at the end of the search
|
|
if input == "" {
|
|
continue
|
|
}
|
|
|
|
// verify the search uses the proper colon-separated syntax
|
|
if !strings.Contains(input, ":") {
|
|
return nil, "column name and value must be separated by a colon"
|
|
}
|
|
|
|
// split the input into field and value
|
|
split := strings.SplitN(input, ":", 2)
|
|
field := split[0]
|
|
value := split[1]
|
|
|
|
// parse the field and value of valid search columns
|
|
switch {
|
|
// --- validate numerical columns
|
|
case slices.Contains(numericalColumns, field):
|
|
|
|
// parse operator and value from input
|
|
operator, number, parseErr := parseSearchOperator(field, value)
|
|
if parseErr != "" {
|
|
return nil, parseErr
|
|
}
|
|
|
|
// validate number is a true number
|
|
numberInt, err := strconv.Atoi(number)
|
|
if err != nil {
|
|
return nil, field + " must be a valid number"
|
|
}
|
|
|
|
// validate and format number to percentage (float from 0-1) for percentage columns
|
|
if slices.Contains(percentageColumns, field) {
|
|
// don't allow values over 100%
|
|
if numberInt > 100 {
|
|
return nil, field + " can't be greater than 100"
|
|
}
|
|
percentage := float64(numberInt) / 100
|
|
// divide value by 100 to convert to decimal, must be formatted to two places
|
|
number = fmt.Sprintf("%1.2f", percentage)
|
|
}
|
|
|
|
// parse and set the operator
|
|
var searchVal OperatorFilter
|
|
if operator == "" {
|
|
// add equals sign when no operator was captured
|
|
searchVal.Operator = "="
|
|
} else {
|
|
searchVal.Operator = operator
|
|
}
|
|
|
|
// set the value
|
|
searchVal.Value = number
|
|
|
|
// assign parsed values to the corresponding field in criteria
|
|
switch field {
|
|
case "count":
|
|
criteria.Count = searchVal
|
|
case "beacon":
|
|
criteria.Beacon = searchVal
|
|
case "subdomains":
|
|
criteria.Subdomains = searchVal
|
|
}
|
|
|
|
// --- validate time columns
|
|
case slices.Contains(timeColumns, field):
|
|
|
|
// parse operator and time string from value
|
|
operator, input, parseErr := parseSearchOperator(field, value)
|
|
if parseErr != "" {
|
|
return nil, parseErr
|
|
}
|
|
|
|
fmt.Println("operator:", operator, "input", input, "parseErr", parseErr)
|
|
|
|
// validate time string is valid
|
|
duration, err := time.ParseDuration(input)
|
|
if err != nil {
|
|
parseErr = field + " must be a valid time in the format '10s', '1.5h', '2h45m', etc. Valid units are 's', 'm', 'h'"
|
|
return nil, parseErr
|
|
}
|
|
|
|
// assign operator to criteria
|
|
if operator == "" {
|
|
// add equals sign when no operator was captured
|
|
criteria.Duration.Operator = "="
|
|
} else {
|
|
criteria.Duration.Operator = operator
|
|
}
|
|
|
|
// convert duration to seconds and assign to criteria
|
|
criteria.Duration.Value = fmt.Sprintf("%.0f", duration.Seconds())
|
|
|
|
// --- validate string columns
|
|
case slices.Contains(stringColumns, field):
|
|
switch field {
|
|
case "src":
|
|
// validate string is IP address
|
|
if _, err := netip.ParseAddr(value); err != nil {
|
|
return nil, "src must be a valid IP address"
|
|
}
|
|
criteria.Src = value
|
|
|
|
case "dst":
|
|
// validate if string is IP address
|
|
if _, err := netip.ParseAddr(value); err != nil {
|
|
// if value is not an IP, check for valid FQDN
|
|
if !util.ValidFQDN(value) {
|
|
return nil, "dst must be a valid IP address or FQDN"
|
|
|
|
}
|
|
criteria.Fqdn = value
|
|
|
|
} else {
|
|
criteria.Dst = value
|
|
}
|
|
case "threat_intel":
|
|
filter, err := strconv.ParseBool(value)
|
|
if err != nil {
|
|
return nil, "threat_intel must be true or false"
|
|
}
|
|
if filter {
|
|
criteria.ThreatIntel = "true"
|
|
} else {
|
|
criteria.ThreatIntel = "false"
|
|
}
|
|
case "sort": // sort:severity-asc
|
|
// split the column from the sort direction
|
|
sortSplit := strings.Split(value, "-")
|
|
if len(sortSplit) != 2 {
|
|
return nil, "sort value must contain one hyphen, in the format sort:<column>-<direction>"
|
|
}
|
|
|
|
// validate sort column and direction
|
|
column := sortSplit[0]
|
|
direction := sortSplit[1]
|
|
|
|
// make sure this column has sorting enabled
|
|
if !slices.Contains(allowedSortColumns, column) {
|
|
return nil, "invalid sort column"
|
|
}
|
|
|
|
// validate sort direction
|
|
if direction != "asc" && direction != "desc" {
|
|
return nil, "sort direction must be either asc or desc"
|
|
}
|
|
|
|
// assign sort column and direction to criteria
|
|
switch column {
|
|
case "severity":
|
|
criteria.SortSeverity = direction
|
|
case "beacon":
|
|
criteria.SortBeacon = direction
|
|
case "duration":
|
|
criteria.SortDuration = direction
|
|
case "subdomains":
|
|
criteria.SortSubdomains = direction
|
|
}
|
|
|
|
case "severity":
|
|
// check if the value is a valid severity
|
|
category := config.ImpactCategory(value)
|
|
err := config.ValidateImpactCategory(category)
|
|
|
|
// if the severity is not critical, check for errors
|
|
// The critical category is not included in the ValidateImpactCategory function because
|
|
// the final score only reaches the Critical category if it exceeds the high threshold
|
|
// via modifiers, so users aren't able to set assign threat categories in the config to critical
|
|
// We also do not want to allow users to filter by the None category
|
|
if err != nil && category != config.CriticalThreat || category == config.NoneThreat {
|
|
return nil, "invalid category, must be 'critical', 'high', 'medium', or 'low'"
|
|
}
|
|
|
|
// assign severity to criteria and set the needed operators for querying
|
|
switch category {
|
|
case config.CriticalThreat:
|
|
criteria.Severity = append(criteria.Severity, OperatorFilter{
|
|
Operator: ">",
|
|
Value: fmt.Sprint(config.HIGH_CATEGORY_SCORE),
|
|
})
|
|
case config.HighThreat:
|
|
criteria.Severity = append(criteria.Severity, OperatorFilter{
|
|
Operator: "<=",
|
|
Value: fmt.Sprint(config.HIGH_CATEGORY_SCORE),
|
|
}, OperatorFilter{
|
|
Operator: ">=",
|
|
Value: fmt.Sprint(config.MEDIUM_CATEGORY_SCORE),
|
|
})
|
|
case config.MediumThreat:
|
|
criteria.Severity = append(criteria.Severity, OperatorFilter{
|
|
Operator: "<",
|
|
Value: fmt.Sprint(config.MEDIUM_CATEGORY_SCORE),
|
|
}, OperatorFilter{
|
|
Operator: ">=",
|
|
Value: fmt.Sprint(config.LOW_CATEGORY_SCORE),
|
|
})
|
|
case config.LowThreat:
|
|
criteria.Severity = append(criteria.Severity, OperatorFilter{
|
|
Operator: "<",
|
|
Value: fmt.Sprint(config.LOW_CATEGORY_SCORE),
|
|
}, OperatorFilter{
|
|
Operator: ">=",
|
|
Value: fmt.Sprint(config.NONE_CATEGORY_SCORE),
|
|
})
|
|
}
|
|
|
|
}
|
|
default:
|
|
return nil, "please reference a valid search column"
|
|
}
|
|
|
|
}
|
|
|
|
return &criteria, ""
|
|
}
|
|
|
|
func parseSearchOperator(field string, value string) (string, string, string) {
|
|
var operator, number, err string
|
|
|
|
// make sure the entire string matches the regex
|
|
if !operatorRegex.MatchString(value) {
|
|
err = fmt.Sprintf("%s value must be %s:<value> or %s:<operator><value>, where <operator> is one of >, <, >=, <=", field, field, field)
|
|
return operator, number, err
|
|
}
|
|
|
|
matches := operatorRegex.FindStringSubmatch(value)
|
|
if matches == nil {
|
|
// no match, bad, this should not happen because we're matching the string above
|
|
err = field + " value is not parseable"
|
|
return operator, number, err
|
|
}
|
|
|
|
// extract operator from regex capture
|
|
operator = matches[operatorRegex.SubexpIndex("operator")]
|
|
|
|
// extract number value from regex capture
|
|
number = matches[operatorRegex.SubexpIndex("value")]
|
|
|
|
return operator, number, err
|
|
}
|