2a5807af83
Fix infinite loop when baselines search fails
...
even after tmax time
2022-09-27 09:20:13 -07:00
2555556301
Removed extra printout and added
...
a clear success/failure printout
2022-09-26 17:06:21 -07:00
124bcd1974
Better error detection/collection for baselines.
...
Previously we treated a failed search in a baseline
as an exception, but this is not necessarily true.
This was causing searches which didn't
immediately have a successful baseline to fail,
which is obviously not correct - we were not
allowing multiple attempts for the data to
be ingested before quitting.
2022-09-26 16:49:05 -07:00
5dbdf3ce38
Fixed logic for handling when a test
...
fails with an uncaught error.
Previously, this could get testing
stuck in an infinite loop.
2022-09-20 09:42:54 -07:00
6b09b24a5a
wait a bit longer to check on ackid
2022-09-19 16:18:59 -07:00
f07ac5187b
Must use shutil.rmtree to
...
delete non-empty directory.
2022-09-19 15:50:54 -07:00
38f135ee81
Remove data after a single detection
...
is tested instead of when all detections
have finished testing.
2022-09-19 15:46:00 -07:00
c97c6a3180
Use the proper port for HEC
...
replay on multiple containers.
Fast fail if there is an actual
error when testing.
2022-09-19 15:30:52 -07:00
46da0af83b
Use hec, with hec setup and
...
token grabbing and indexer
acknowledgement, to replay
data. Test seaches, to include
baselines, using exponential
backoff algorithm. This can result
in a huge testing speedup.
2022-09-19 14:47:12 -07:00
f3f6e9649a
Always download the data file when it's
...
linked via a URL
2022-09-13 14:39:18 -07:00
60e8c13ddc
Tests are now running successfully and a basic
...
output file is being generating. Still needs lots
of smoketesting, but the new workflow is looking
good!
2022-09-12 16:05:30 -07:00
ac506cde60
Initial testing workflow looks
...
like it is correct. However,
errors during
2022-09-10 09:10:01 -07:00
e8e8b89f6c
fix reference to attack_data
...
of dict type instead of AttackData
type
2022-09-09 14:54:40 -07:00
ad8be14783
Fix how data is fetched and make
...
sure to move on to another test
after a failure
2022-09-09 14:52:38 -07:00
5b04f3811e
Better organization of objects
...
and resolving a circular import
2022-09-09 14:31:22 -07:00
7ce4a575f4
Still a lot more changes, especially
...
to how results are returned. Have not
tested it running yet.
2022-09-09 14:20:37 -07:00
8eef2e3e64
More refactoring of how tests are
...
executed as we move from dicts
to objects. Makes changes, maintenance,
etc so much easier moving forward.
2022-09-09 11:15:46 -07:00
bd4ab39e3c
More huge updates to support testing
...
multiple tests per file. dicts are getting
hard to manage - it may make more
sense to turn all of this state into
objects instead.
2022-09-07 17:06:38 -07:00
d317ffa394
Huge changes to restructure how tests
...
are executed. Progress towards
allowing multiple tests per detection
to be executed.
2022-09-07 14:24:49 -07:00
82f9945128
Changes to testing_service.py to support
...
multiple tests per test file. Still have not tested or
worked out how this will all be reported
in the summary, though.
2022-09-06 17:07:27 -07:00
71b7c5d45c
Forgot to specify the host that data
...
was being uploaded from. This means
that delete does not work as expected,
causing a large number of tests to fail
and slower searches.
2022-07-11 14:06:01 -07:00
c98d7b6855
Fix an issue where trying to download a file from attack_data that returns a 404 fails too late in the process, giving a nondescriptive error message and resulting in a bad filename being included in the detection failure manifest.
2022-06-21 12:52:32 -07:00
1e9b95542d
Missing parenthesis added
2022-06-08 15:30:49 -07:00
8eb675522e
Moved printout of delete status
...
updates to a better location and
condensed multiple printouts for
multiple indices to 1.
2022-06-08 15:21:54 -07:00
5e36835ce9
Moving default data index
...
definition to splunk_sdk file.
2022-06-08 14:57:33 -07:00
d3da287905
Added support for specifying a custom index
...
to delete after a test in each attack_data
dataset in the test file. It is important to
have this level of control (per-dataset), not
just per test or per-file!
2022-06-06 16:08:06 -07:00
cff7ba417a
cleaning up old CI jobs
2022-03-22 15:37:49 -04:00
pyth0n1c