Files
Naomi Kramer b923c39e2d Update configuration (#78)
* Update on config structure, functionality, and tests

Co-Authored-By: Naomi Kramer <naomiagoddard@gmail.com>

* Extend subnet type to read/write from db, update tests

Co-Authored-By: Liza Tsibur <liza@activecountermeasures.com>

* updated read file config test and subnet tests

* fixed config and util tests, updated subnet related functions

Co-Authored-By: Naomi Kramer <naomiagoddard@gmail.com>

* Remove error return from GetDefaultConfig

Co-Authored-By: Liza Tsibur <liza@activecountermeasures.com>

* added json tags to database struct

* Updated beacon weights validation for config

* updates to score thresholds validation tags

* changes to config subnet validation and testing

* Update subnet.go

* Write missing host entries to http to populate http_proto

* Updating some fields to uint64

* WIP update some field types

* Update zeek count types and fix tests

* Add clickhouse credentials

* Misc fixes

* Update pointer

* Add ability to mark datasets as sample datasets

* fix column name

* Fix datasets exiting import if hour is empty

* Fix zeek count parsing from TSV files

* Remove storing dns conns in arrays, Fix historical first seen dns lag

* Remove unused columns

* Update config.hjson

* Update config.hjson

* updated impact category score functions to use float64

Co-Authored-By: Naomi Kramer <naomiagoddard@gmail.com>

* Update subnet.go

* Store import version in imports table

* Fix duplicated SNI/IP long connections

* Update subnet_test.go

* Cleanup output

* Rolling files updates (#39)

* Limit number of days to import for rolling datasets

* Fix breaking imports when import was interrupted

* Remove debug output

---------

Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com>

* Omit parts of env from output

* Set max for threat intel datasize

* Remove SELinux neutering for QA

* Add network size column

* Fix http_proto for missing host, update tests for missing host fixes

* Add online feeds to default config

* Update sshprep (#45)

* Update sshprep

Co-Authored-By: William Stearns <3538265+william-stearns@users.noreply.github.com>

* Update sshprep

Add Bradley's suggestion of using head -1 to limit to a single address.

---------

Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com>
Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com>

* Installer Behavior Tweaks (#41)

* Add --yes flag to add-apt-repository command

* Add missing sudo flags, make sure we're using the SUDO variable instead

* Add ability to perform zone transfers (#48)

* Store zone transfer records

Co-Authored-By: moth <25512187+0x6d6f7468@users.noreply.github.com>

* Update config

* Add tests

* Tests, connectivity test

* Update tests

---------

Co-authored-by: moth <25512187+0x6d6f7468@users.noreply.github.com>

* Support RedHat/RHEL as a valid target (#47)

* Update sshprep

Co-Authored-By: William Stearns <3538265+william-stearns@users.noreply.github.com>

* Supporrt RedHat/RHEL as a valid target

---------

Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com>
Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com>
Co-authored-by: moth <moth@blackhillsinfosec.com>

* Fix tests (#49)

* Fix tests

* Update WalkFiles to use UTC

* fixed issue with rolling datasets over 24hours old not getting historical first seen timestamp set (#52)

* Change values from float32 to float64 (#50)

* Switch float32 to float64

* Update threat category calculation to match CalculateBucketedScore (#51)

---------

Co-authored-by: Liza Tsibur <liza@activecountermeasures.com>

* Bump max query execution time default value

* Use string instead of error for ZoneTransferConnectivityErrors struct fields (#61)

* Upgrade Golang to version 1.24 (#59) (#60)

* Replace get_url with shell and curl (#58)

* Update sshprep

Co-Authored-By: William Stearns <3538265+william-stearns@users.noreply.github.com>

* Replace get_url with shell and curl

* Use get_url by default, fall back to curl if it fails

---------

Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com>
Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com>
Co-authored-by: moth <moth@blackhillsinfosec.com>

* add automated log transfer, AC-Hunter issue 135 (#62)

* Update sshprep

Co-Authored-By: William Stearns <3538265+william-stearns@users.noreply.github.com>

* add automated log transfer, PR135

* cron requires non-executable permission

* Specify suggested YAML plugin and config in VSCode workspace

* Linting and light cleanup

* Update generate_installer.sh

Download zeek_log_transport.sh to send to the sensor.

* Create cron file if remote zeek installation

* Only run zeek log import steps for remote sensor installations

---------

Co-authored-by: Naomi Kramer <naomi@activecountermeasures.com>
Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com>
Co-authored-by: moth <moth@blackhillsinfosec.com>

* Temporarily disable RITA/Zeek log transport until installer is modular (#66)

* Uniform -y flag usage for repo management/package installation; Uniform SUDO variable usage (#68)

* Resolve Installer Side Effects and Formalize RHEL Support (#73)

* Add missing necessary wildcards for RHEL versions

* Remove Ansible task replacing python3-requests to avoid RHEL distro installation side effects

* Update supported distros in README

* Update scoring defaults

* Resolve Ansible Reboot Errors (#75)

* Clean up conditionals; Fix reboot step for Ubuntu

* Suppress erroneous error output on RPM systems, ignore errors on reboot necessity checks

* Ignore missing host rows for openhttp (#76)

* Fix integration tests due to prevalence (#77)

---------

Co-authored-by: Liza Tsibur <liza@activecountermeasures.com>
Co-authored-by: moth <moth@blackhillsinfosec.com>
Co-authored-by: William Stearns <william.l.stearns@gmail.com>
Co-authored-by: William Stearns <3538265+william-stearns@users.noreply.github.com>
Co-authored-by: moth <25512187+0x6d6f7468@users.noreply.github.com>
2025-09-22 16:21:05 -04:00

104 lines
4.1 KiB
Go

package analysis
import (
"log"
"testing"
"github.com/activecm/rita/v5/config"
"github.com/joho/godotenv"
"github.com/spf13/afero"
"github.com/stretchr/testify/require"
)
func TestMain(m *testing.M) {
err := godotenv.Load("../.env")
if err != nil {
log.Fatal("Error loading .env file")
}
m.Run()
}
/* **** THREAT SCORE BUCKETS ****
Each Threat Indicator's score is placed into a categorical severity bucket.
Since the units of measurement for each threat indicator varies, we must
place the scores into buckets based on predefined thresholds for each bucket.
Each bucket has a set, non-configurable score that it applies to.
None: 0%
Low: 20%-40%
Medium: 41%-60%
High: 61%-80%
*/
func TestCalculateBucketedScore(t *testing.T) {
// verify that the score is greater than zero when the none threshold is zero
// (and the value is greater than zero)
score := calculateBucketedScore(1, config.ScoreThresholds{Base: 0, Low: 5, Med: 10, High: 15})
require.Greater(t, score, float64(0), "score must be greater than zero when the base threshold is zero & the value is greater than zero")
// verify that the score is 20% if the none threshold and the value are zero
// this allows configuration for any positive integer to score at least 20%
score = calculateBucketedScore(0, config.ScoreThresholds{Base: 0, Low: 5, Med: 10, High: 15})
require.InDelta(t, 0.2, score, 0.0001, "score must be 0.2 if the base threshold and the value are zero")
cfg, err := config.ReadTestFileConfig(afero.NewOsFs(), "../config.hjson")
require.NoError(t, err)
type testCase struct {
Name string
Thresholds config.ScoreThresholds
}
testCases := []testCase{
{Name: "C2 Over DNS", Thresholds: cfg.Scoring.C2ScoreThresholds},
{Name: "Long Connections", Thresholds: cfg.Scoring.LongConnectionScoreThresholds},
{Name: "Beacons", Thresholds: cfg.Scoring.Beacon.ScoreThresholds},
}
for _, test := range testCases {
base := float64(test.Thresholds.Base)
low := float64(test.Thresholds.Low)
medium := float64(test.Thresholds.Med)
high := float64(test.Thresholds.High)
t.Run(test.Name, func(t *testing.T) {
score = calculateBucketedScore(base-1, test.Thresholds)
require.InDelta(t, 0, score, 0.00001, "score should be zero when value is lower than the None bucket threshold")
// verify score matches the low threshold bucket score
score = calculateBucketedScore(base, test.Thresholds)
require.InDelta(t, .20, score, 0.001, "score must match the base threshold bucket score")
score = calculateBucketedScore(low-1, test.Thresholds)
require.InDelta(t, .3995, score, 0.1, "score should be very close to the low threshold score if it is almost at the medium threshold")
score = calculateBucketedScore(low, test.Thresholds)
require.InDelta(t, .40, score, 0.001, "score must match the low threshold bucket score")
betweenLowAndMedium := low + ((medium - low) / 2)
score = calculateBucketedScore(betweenLowAndMedium, test.Thresholds)
require.InDelta(t, .50, score, 0.1, "score should interpolate between the low and medium bucket")
score = calculateBucketedScore(medium-1, test.Thresholds)
require.InDelta(t, .5995, score, 0.1, "score should be very close to the medium threshold score if it is almost at the high threshold")
score = calculateBucketedScore(medium, test.Thresholds)
require.InDelta(t, .60, score, 0.001, "score must match the medium threshold bucket score")
betweenMediumAndHigh := medium + ((high - medium) / 2)
score = calculateBucketedScore(betweenMediumAndHigh, test.Thresholds)
require.InDelta(t, .70, score, 0.1, "score should interpolate between the medium and high bucket")
score = calculateBucketedScore(high-1, test.Thresholds)
require.InDelta(t, .7995, score, 0.1, "score should be very close to the high threshold score if it is almost at the high threshold")
score = calculateBucketedScore(high, test.Thresholds)
require.InDelta(t, .80, score, 0.001, "score must match the high threshold bucket score")
score = calculateBucketedScore(high*2, test.Thresholds)
require.InDelta(t, .80, score, 0.001, "score must match the high threshold bucket score even if the value is larger than the high threshold")
})
}
}