10390 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
pyth0n1c	afbc033601	Wait some time after ready before starting all of the tests to allow the container to settle.	2021-10-26 16:58:24 -07:00
pyth0n1c	710efec1a0	Tweak how long we wait before starting tests with the containers.	2021-10-26 16:19:37 -07:00
pyth0n1c	5fc1b7585b	Fixed the last source to . changes for building slim. Also added the number of containers argument to the Synchronization object.	2021-10-26 15:55:07 -07:00
pyth0n1c	b97fb13555	Rolling back some changes that introduced slim installation errors	2021-10-26 15:25:26 -07:00
pyth0n1c	673c95673c	Fixed slim install	2021-10-26 14:47:01 -07:00
pyth0n1c	549bcac8a7	Changes to support Ubuntu, which doesn't have curl installed by default, and moving to python3 for slim.	2021-10-26 14:33:01 -07:00
pyth0n1c	fd9e91d569	Write out a summary csv and write environment iformation, to include datetime, splunk version, and installed splunk apps, to the csv files.	2021-10-25 14:28:06 -07:00
pyth0n1c	83d6869c6c	Better support for different types of structured output files, including a summary of all the detections before they are even run.	2021-10-25 11:24:16 -07:00
pyth0n1c	0846bb6d0b	Putting download attack data into a 'temp dir' and removing attack data as soon as it's used. We need to do this when we're testing on CI/CD, otherwise we will run out of space. Should keep/remove attack data be an option instead of forced?	2021-10-22 16:42:53 -07:00
pyth0n1c	194de0c28a	Added ability to test individual detection by specifying them on the command line.	2021-10-22 15:41:18 -07:00
pyth0n1c	53dedd468f	Big changes to make the tool much more usable. Actually useful for testing, but no nice output files just yet.	2021-10-22 12:57:38 -07:00
pyth0n1c	c5b5dfac88	Lots more changes - adding more error checking, typing to more functions, and better logging for testing detections. Initial cut at fixing up issues involving bad return types. Still had not been run/tested yet.	2021-10-21 17:06:35 -07:00
pyth0n1c	365bcc82ac	More cleanup and removal of dead code. Created an object to synchronize all the threads and their status instead of having a massive number of arguments to each thread. Includes a synchronization primitive.	2021-10-21 15:32:57 -07:00
pyth0n1c	7ba61bc2b5	Generate good password for a container instead of always using a static password. Note that a password can also be supplied on the command line.	2021-10-21 12:22:28 -07:00
pyth0n1c	10d43aa599	Added functionality and checks to remove the security_content directory or persist the security_content directory. This is useful because, on slow connections, it makes running subsequent tests A LOT faster. It is also faster on a fast connection. Finally, and most importantly, it lets you easily persist and re-test changes that you've made to detections.	2021-10-20 18:17:07 -07:00
pyth0n1c	60262a57f8	Added some more command line arguments to speed up testing and did some more smoketesting.	2021-10-20 17:33:26 -07:00
pyth0n1c	4e52891726	Better parsing of files to test, better error handling, more code reuse.	2021-10-20 17:06:44 -07:00
pyth0n1c	505e2f7986	Cleanup and restructuring of main logic. Better prints, error handling, and readability.	2021-10-20 15:13:50 -07:00
pyth0n1c	9995e9123d	Added some additional arguments to control which detections we will test.	2021-10-19 17:14:30 -07:00
pyth0n1c	9e76ab75fc	Working better during testing, but still needs a huge amount of cleanup. Lots of dead code and magic strings.	2021-10-19 16:49:43 -07:00
pyth0n1c	75fe6a9f61	More progress. Now build and upload apps from local file system as well as install from splunkbase.	2021-10-15 13:55:46 -07:00
pyth0n1c	b607f8a99d	Some small config updates	2021-10-13 12:31:02 -07:00
pyth0n1c	1a131ead59	More changes for testing. Prep for long-term test of all detection with results output to file.	2021-10-12 17:22:10 -07:00
pyth0n1c	2f06facd23	Lots of testing changes.... lots of progress. Need to clean up and test entire set of detections properly with sleeps and diagnose how long sleep should be or if there is a way to block while input data is processed entirely.	2021-10-11 17:05:28 -07:00
pyth0n1c	06a9c0f15e	Lots more changes to how we run. Now based off of the splunk/splunk container on docker hub as much as possible. We use ENV arguments to install required apps. Still only at 50% pass rate, so we need more troubleshooting to figure out why.	2021-10-11 10:23:42 -07:00
pyth0n1c	e0961ad0d3	Still some issues when running more than one container in parallel. Might be a management port issue, but needs debugging.	2021-09-20 17:21:46 -07:00
pyth0n1c	f30a4c6997	Re-added rebuild code for full test.	2021-09-20 11:05:17 -07:00
pyth0n1c	0cd2cd850f	Duplicate folder names were clobbering each other. Also commented around initial steps to speed up testing time. We will re use the docker containers that we initially built each time for testing.	2021-09-20 11:01:06 -07:00
pyth0n1c	ba78e372e2	Changed a port back to the original since it's run inside the docker container as part of a playbook	2021-09-17 18:12:02 -07:00
pyth0n1c	2a3e220249	Made the splunk management port dynamic for uploading replay data.	2021-09-17 17:51:27 -07:00
pyth0n1c	5588eeabe1	Fixed ugly multi line comment	2021-09-17 17:35:50 -07:00
pyth0n1c	94f05e573c	Added some error handling so that the show can go on while we debug. Fixed another bad variable naming error	2021-09-17 17:33:20 -07:00
pyth0n1c	ebb00af725	Forgot to declare results_queue before using it.	2021-09-17 16:58:14 -07:00
pyth0n1c	4d1c456a33	Removed some aws calls which should no longer be made	2021-09-17 16:49:05 -07:00
pyth0n1c	e5c87027c1	Lots more changes building out the skeleton of the docker testing framework. Needs testing, breakup into simpler files, documentation, etc. But a good start.	2021-09-17 16:45:05 -07:00
pyth0n1c	dc8b704332	Initial changes and testing for local dockerized detection testing service	2021-09-17 15:14:17 -07:00
Michael Haag	b6db6f357b	Merge pull request #1667 from splunk/add_dataset_url ... adding dataset url	2021-09-17 06:24:41 -06:00
Jose Enrique Hernandez	aec4f7c772	adding dataset url	2021-09-15 17:34:51 -04:00
pyth0n1c	3f5b996824	Merge pull request #1664 from splunk/TR-851 ... removed submodule, no need for additonal detection testing run. Semgrep error is due to issue in semgrep repo	2021-09-15 11:36:01 -07:00
Lou Stella	e3a84e4840	removed submodule	2021-09-15 12:54:05 -05:00
mvelazco	1fde288203	Merge pull request #1662 from splunk/AD_Discovery_TR-789_5 ... May The Haag Be With You	2021-09-14 15:34:29 -04:00
root	575377d312	Added detection testing service results inPowerShell Get LocalGroup Discovery	2021-09-14 18:46:53 +00:00
Detection Testing Service	6bed1bdc71	Merge branch 'AD_Discovery_TR-789_5' of https://github.com/splunk/security_content into AD_Discovery_TR-789_5	2021-09-14 18:20:58 +00:00
root	70dfb3b5d1	Added detection testing service results inWmic Group Discovery	2021-09-14 18:20:57 +00:00
Detection Testing Service	1a5e50fbec	Merge branch 'AD_Discovery_TR-789_5' of https://github.com/splunk/security_content into AD_Discovery_TR-789_5	2021-09-14 18:14:29 +00:00
root	eaacec47db	Added detection testing service results inPowershell Get LocalGroup Discovery with Script Block Logging	2021-09-14 18:14:29 +00:00
Detection Testing Service	84f5346fdf	Merge branch 'AD_Discovery_TR-789_5' of https://github.com/splunk/security_content into AD_Discovery_TR-789_5	2021-09-14 18:02:07 +00:00
root	c514bf1361	Added detection testing service results inNet Localgroup Discovery	2021-09-14 18:02:07 +00:00
Detection Testing Service	5cf90f60b2	Merge branch 'AD_Discovery_TR-789_5' of https://github.com/splunk/security_content into AD_Discovery_TR-789_5	2021-09-14 17:55:42 +00:00
root	7ccd84ccfd	Added detection testing service results inGet WMIObject Group Discovery with Script Block Logging	2021-09-14 17:55:41 +00:00