f19e5c01fd
Much better support for persist_security_content with warnings if local changes will be overwritten. Much better checking if a detection search has failed - the search will be run again after some more time is allowed to pass. This makes sure that all the data has had time to be properly ingested. There is still a potential issue when checking out different branches which will need some additional testing.
2022-02-16 17:31:01 -08:00
0221f974e1
Now, if a test fails, the workflow will
...
fail. Also, significant updates to how
detections to be tested are checked out.
This first commit is especially to test
github Actions updates.
2022-02-04 09:50:14 -08:00
f793a802ec
Changes and updates to support better, more reliable
...
checkouts of PRs from remote forks and better
ESCU app building. Still in testing phases.
2022-02-03 17:15:45 -08:00
5ecb9f5fbd
Tests were failing on GH Actions that should not have been failing. This is likely due to not enough time waiting for indexing to fail. Doubled the time we wait between indexing complete check. Also moved from hourly tests of all detections to daily.
2022-01-13 15:50:07 -08:00
cc90aca4b9
Fixed spacing in printouts for
...
detection-testing.yml
2022-01-13 10:35:23 -08:00
928797252d
Changed from
...
github.event.action to
github.event_name
2022-01-13 10:32:49 -08:00
da5e00e580
Added spaces to fix format of
...
github.event.action
2022-01-13 10:25:07 -08:00
77b5fde422
Fixed scheduled action for nightly test.
...
Now, it will hopefully only run the test of
the entire repo on the SCHEDULED
github.event.action, not on scheduled
event of any commit to the develop branch.
We have changed the schedule interval to
hourly for testing purposes today and will
change it back to nightly after today's testing.
2022-01-13 10:22:25 -08:00
8d669b66f8
Removing previous file that was added accidentally in last commit. Updating the detection-testing.yml file to remove ls of non-existant directory as intended
2022-01-12 16:17:38 -08:00
2cb3390164
Migrated the docker detection testing from
...
build-and-validate.yml into detection-testing.yml
workflow. Removed the unused AWS batch
testing workflow. This makes much more sense
2022-01-12 16:05:22 -08:00
c9d0e72cf3
removed accidental brackets from branch name in
...
CI testing branch name.
2022-01-12 14:04:38 -08:00
588eaad92a
Accidentally hardcoded the branch name for testing.
...
Fixed.
2022-01-12 13:44:01 -08:00
4d9349084e
Changed to run nightly on develop and changes on other branches for pushes and PRs.
2022-01-11 16:37:03 -08:00
7b3029b187
Removed splunkbase-only test_config file. Pushing to run a test on all. Will push again for changes only.
2022-01-11 16:35:19 -08:00
2574fb2fea
Hardcoding the branch to develop for CI testing.
2021-12-22 13:19:05 -08:00
04c526ba6d
Changed the wrong line to trigger a test of everything. Trying again.
2021-12-21 11:17:09 -08:00
5f972f734f
Re-push to test everything with 2 containers per GH Actions machine.
2021-12-21 11:00:11 -08:00
5a554f3b5b
Updated paths in workflows that changed due to
...
relocating the automated_detection_testing
folder in bin/
2021-12-08 14:09:22 -08:00
04ba2b5ae6
Merge branch 'develop' into DetectionTestingDocker
2021-12-08 13:04:28 -08:00
8f55eb5848
Proper summary.json files are now always generated after a run of the CI. It does not require a separate run of summarize_json.py. Becuase the CI runs different jobs, though, it is still required to run this explicitly in the CI. Also fixed an issue where searches that resulted in an error had incorrect json output due to improper python error handling.
2021-12-08 12:00:51 -08:00
14e548c65a
More changes to CI. Fixed a bash typo and removed the docker-detection-testing.yml since it has been integrated into build-and-validate.
2021-12-06 14:54:58 -08:00
3de3ffeada
Brace in wrong location - around non environment variable
2021-12-06 14:44:27 -08:00
2981a9c592
added a check to see if we are running on develop, meaning that it is a nightly job. Also, added a cron job to fun of 04:44 UTC. GitHub recommends against the top of the hour because load is higher at that time.
2021-12-06 14:37:24 -08:00
652355a124
Typo in branch name caused an error during checking.
2021-12-06 11:51:14 -08:00
9be4d1a4eb
Large changes to json validation for local_apps. json validation was wrong. also changed CI to run on push and on PR using the new CI job! Note that it will still require lots of testing.
2021-12-06 11:41:35 -08:00
6457b505db
When there is an error checking out a git PR. If the PR number is bad, we will exit. Also updated the CI/CD to test all of develop again. We will look at the results after we have increased the wait time after loading data a bit.
2021-12-01 15:58:31 -08:00
531d550b3b
Just change the workflow to explicitly test the RegistryDetectionFixes branch.
2021-12-01 14:44:51 -08:00
c085327a72
Change the detection testing mode to all for a test of everything.
2021-11-30 19:33:46 -08:00
f1a10f0d65
Upload the updated test file so we can replicate the test in the future.
2021-11-30 10:36:12 -08:00
eb6378e805
Made some changes to properly supporting testing changed test file. Had some trouble before locating the appropriate detection file. Also throw an error if we try to test a file that was changed and is in the experimental path. Set up to run a test of the DetectionBaselineFixes branch to test these changes in GitHub Actions.
2021-11-30 09:52:36 -08:00
868c98082b
Small change to test changes on a specific branch instead of develop.
2021-11-29 16:49:06 -08:00
2b3fd66bb3
Run a test on everything, saving the intermediate files to help debugging.
2021-11-29 11:54:26 -08:00
fff797358c
Run a handlful of tests on push to produce output artifacts.
2021-11-29 11:34:59 -08:00
95e7169e05
fix quote
2021-11-29 10:53:29 -08:00
e8fb9578ea
Change types of quotes around schedule
2021-11-29 10:51:44 -08:00
38ad36c296
Run at 55th minute each hour for testing.
2021-11-29 10:49:54 -08:00
e1844e0333
removed devsecops
2021-11-25 08:55:22 +01:00
afb07b6714
Run it on push too for an immediate test
2021-11-24 17:14:33 -08:00
b94ef641ec
Changed the detection testing to run on all, develop instead of changes to a different branch. This will collect some data for us over the next few days.
2021-11-24 17:13:41 -08:00
ae3e8f1b75
slim failed to install, causing escu app install to fail.
2021-11-24 14:56:38 -08:00
aa985d5077
Made some small changes to printed strings. Fixed summarize_json missing python library error by sourcing virtualenv set up in previous step.
2021-11-24 14:42:18 -08:00
9517171204
Small change to the output format of the test-config file that is generated at runtime to accurately describe the current test. Also added install of requirements.txt to final summary step.
2021-11-24 14:20:46 -08:00
88977a9fb8
Changes to template and summary to support passing a commit_hash parameter, not just a branch parameter, so that even old tests can be replicated.
2021-11-24 13:58:32 -08:00
4f6be10581
Testing with the proper mode for a specific branch.
2021-11-24 12:00:02 -08:00
f32f1bed2a
More verbose debug messages in summarize_json.
2021-11-24 11:45:52 -08:00
ccc29005fa
Still diagnosing path issues for summarize.
2021-11-24 11:12:19 -08:00
54a34608fe
Fixed results paths once more for summary.
2021-11-24 10:36:33 -08:00
5012b0f766
Another run to check and see if the summarization works and confirm the directory structure of the artifacts.
2021-11-24 10:10:27 -08:00
3cd3ca6692
Another CI change to list directory structure during summarize step.
2021-11-24 09:54:27 -08:00
6b292ba2c1
upload artifacts even if summarize fails.
2021-11-23 17:13:44 -08:00
pyth0n1c