 pyth0n1c
|
e453ffefb8
|
More verbose error message if the Git checkout fails.
|
2021-12-06 11:55:08 -08:00 |
|
|
pyth0n1c
|
652355a124
|
Typo in branch name caused an error during checking.
|
2021-12-06 11:51:14 -08:00 |
|
|
pyth0n1c
|
9be4d1a4eb
|
Large changes to json validation for local_apps. json validation was wrong. also changed CI to run on push and on PR using the new CI job! Note that it will still require lots of testing.
|
2021-12-06 11:41:35 -08:00 |
|
|
pyth0n1c
|
17726fcdff
|
Running a test with the old sysmon to compare the results.
|
2021-12-02 10:00:01 -08:00 |
|
|
pyth0n1c
|
6457b505db
|
When there is an error checking out a git PR. If the PR number is bad, we will exit. Also updated the CI/CD to test all of develop again. We will look at the results after we have increased the wait time after loading data a bit.
|
2021-12-01 15:58:31 -08:00 |
|
|
pyth0n1c
|
531d550b3b
|
Just change the workflow to explicitly test the RegistryDetectionFixes branch.
|
2021-12-01 14:44:51 -08:00 |
|
|
pyth0n1c
|
859bd9c6da
|
Some small changes dealing with errors during a test.
|
2021-12-01 11:00:04 -08:00 |
|
|
pyth0n1c
|
c085327a72
|
Change the detection testing mode to all for a test of everything.
|
2021-11-30 19:33:46 -08:00 |
|
|
pyth0n1c
|
c9c2f7c3e3
|
Initial support for testing if a container crashes or fails to start during setup. In that case, the whole test will finish, the other containers will be stopped, and we will fail.
|
2021-11-30 19:32:52 -08:00 |
|
|
pyth0n1c
|
fee83d2cfd
|
Check for splunkbase_username and splunkbase_password immediately if Splunkbase apps are to be installed and fail with a descriptive error if they are not provided.
|
2021-11-30 14:22:34 -08:00 |
|
|
pyth0n1c
|
9700571e0c
|
Set required version of jsonschema. Added an --interactive_failure command line option. Fixed a typo and removed non required jsonschema library
|
2021-11-30 13:43:25 -08:00 |
|
|
pyth0n1c
|
f1a10f0d65
|
Upload the updated test file so we can replicate the test in the future.
|
2021-11-30 10:36:12 -08:00 |
|
|
pyth0n1c
|
e759f01e0c
|
Better command line summary of pass and fail for summarize_json.py
|
2021-11-30 10:16:05 -08:00 |
|
|
pyth0n1c
|
eb6378e805
|
Made some changes to properly supporting testing changed test file. Had some trouble before locating the appropriate detection file. Also throw an error if we try to test a file that was changed and is in the experimental path. Set up to run a test of the DetectionBaselineFixes branch to test these changes in GitHub Actions.
|
2021-11-30 09:52:36 -08:00 |
|
|
pyth0n1c
|
868c98082b
|
Small change to test changes on a specific branch instead of develop.
|
2021-11-29 16:49:06 -08:00 |
|
|
pyth0n1c
|
180785c7d4
|
Certain types of error would have missing fields in the results json, causing the summarization code to crash. Fixed that. Also sorted the output of the summarization to made it easier to look at a glance. Errors and failures will be at the top and the rest will be sorted alphabetically.
|
2021-11-29 14:40:17 -08:00 |
|
|
pyth0n1c
|
2b3fd66bb3
|
Run a test on everything, saving the intermediate files to help debugging.
|
2021-11-29 11:54:26 -08:00 |
|
|
pyth0n1c
|
fff797358c
|
Run a handlful of tests on push to produce output artifacts.
|
2021-11-29 11:34:59 -08:00 |
|
|
pyth0n1c
|
95e7169e05
|
fix quote
|
2021-11-29 10:53:29 -08:00 |
|
|
pyth0n1c
|
e8fb9578ea
|
Change types of quotes around schedule
|
2021-11-29 10:51:44 -08:00 |
|
|
pyth0n1c
|
38ad36c296
|
Run at 55th minute each hour for testing.
|
2021-11-29 10:49:54 -08:00 |
|
|
pyth0n1c
|
afb07b6714
|
Run it on push too for an immediate test
|
2021-11-24 17:14:33 -08:00 |
|
|
pyth0n1c
|
b94ef641ec
|
Changed the detection testing to run on all, develop instead of changes to a different branch. This will collect some data for us over the next few days.
|
2021-11-24 17:13:41 -08:00 |
|
|
pyth0n1c
|
827cc97cdc
|
Better support for arg validation and command line arguments. Print out the password for the container, if the flag exists to print it out, to enable interactive testing. A few other format updates.
|
2021-11-24 17:06:07 -08:00 |
|
|
pyth0n1c
|
abd568f7d7
|
Small update to summarize_json.py to set show_splunk_app_password to True when generatingf the detection_failure_manifest.json file. This way, you can run that file directly as the configuration without making any other changes or passing command line values and get right into debugging everything
|
2021-11-24 15:57:13 -08:00 |
|
|
pyth0n1c
|
eb01127d03
|
Fixed key error in summarize.json. Had called detection_list detections_list, so it was failing jsonschema validation.
|
2021-11-24 15:25:24 -08:00 |
|
|
pyth0n1c
|
ae3e8f1b75
|
slim failed to install, causing escu app install to fail.
|
2021-11-24 14:56:38 -08:00 |
|
|
pyth0n1c
|
aa985d5077
|
Made some small changes to printed strings. Fixed summarize_json missing python library error by sourcing virtualenv set up in previous step.
|
2021-11-24 14:42:18 -08:00 |
|
|
pyth0n1c
|
9517171204
|
Small change to the output format of the test-config file that is generated at runtime to accurately describe the current test. Also added install of requirements.txt to final summary step.
|
2021-11-24 14:20:46 -08:00 |
|
|
pyth0n1c
|
88977a9fb8
|
Changes to template and summary to support passing a commit_hash parameter, not just a branch parameter, so that even old tests can be replicated.
|
2021-11-24 13:58:32 -08:00 |
|
|
pyth0n1c
|
4f6be10581
|
Testing with the proper mode for a specific branch.
|
2021-11-24 12:00:02 -08:00 |
|
|
pyth0n1c
|
f32f1bed2a
|
More verbose debug messages in summarize_json.
|
2021-11-24 11:45:52 -08:00 |
|
|
pyth0n1c
|
ccc29005fa
|
Still diagnosing path issues for summarize.
|
2021-11-24 11:12:19 -08:00 |
|
|
pyth0n1c
|
54a34608fe
|
Fixed results paths once more for summary.
|
2021-11-24 10:36:33 -08:00 |
|
|
pyth0n1c
|
5012b0f766
|
Another run to check and see if the summarization works and confirm the directory structure of the artifacts.
|
2021-11-24 10:10:27 -08:00 |
|
|
pyth0n1c
|
3cd3ca6692
|
Another CI change to list directory structure during summarize step.
|
2021-11-24 09:54:27 -08:00 |
|
|
pyth0n1c
|
6b292ba2c1
|
upload artifacts even if summarize fails.
|
2021-11-23 17:13:44 -08:00 |
|
|
pyth0n1c
|
a302d952b4
|
tiny ci change to test everything overnight
|
2021-11-23 17:12:25 -08:00 |
|
|
pyth0n1c
|
ae7fb14085
|
Updated summarize_json.py to provide a total success/failure field and also return a nonzero return code if at least one test fails.
|
2021-11-23 16:38:22 -08:00 |
|
|
pyth0n1c
|
1a39321be6
|
Small typo in artifact filename.
|
2021-11-23 15:54:12 -08:00 |
|
|
pyth0n1c
|
1afbaa3a1d
|
Fixed bad value substitution into the schema coming from command line arguments.
|
2021-11-23 15:35:05 -08:00 |
|
|
pyth0n1c
|
451d2500db
|
Fixed overwriting mode with null if nothing is passed on the command line.
|
2021-11-23 15:22:11 -08:00 |
|
|
pyth0n1c
|
b0e7c33c4d
|
Changing mode for CI PR test from selected to changes.
|
2021-11-23 15:14:49 -08:00 |
|
|
pyth0n1c
|
1015abc30a
|
Finally ready to try the full integration test after modifying the docker-detection-testing.yml with new names, paths, and arguments. Fingers crossed...
|
2021-11-23 15:10:28 -08:00 |
|
|
pyth0n1c
|
fab525a682
|
Added support for more command line arguments for number containers and mode and branch.
|
2021-11-23 14:48:37 -08:00 |
|
|
pyth0n1c
|
dab746bb0c
|
Save out a json file showing the config, to include the command line arguments with credentials removed, whenever a test is actually run. This makes it trivial to reproduce the test run on another machine or again on the same machine.
|
2021-11-23 14:30:54 -08:00 |
|
|
pyth0n1c
|
914f49d6f7
|
Delete intermediate testing artifacts/container config files in CI. Better job packaging all local apps, not just escu, into the same folder for --mock.
|
2021-11-23 11:59:01 -08:00 |
|
|
pyth0n1c
|
3cd1e42ada
|
Huge refactor is almost complete. Able to launch and run multiple containers now as intended
|
2021-11-22 16:36:21 -08:00 |
|
|
pyth0n1c
|
3f1b025908
|
Mock is now working and splits the tests as intended. Appropriate mock config.json files are created, validated, and written.
|
2021-11-19 17:07:27 -08:00 |
|
|
pyth0n1c
|
6a9b5504aa
|
Converted the app listings in the jsonschema from arrays to dicts. they are easier to search for apps that way. also some more testing of the escu app building and persist_security_content. looking good.
|
2021-11-19 16:06:04 -08:00 |
|