admin/splunk-security_content
1
0
Fork 0
mirror of https://github.com/splunk/security_content synced 2026-06-08 17:32:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

updating with speec3.0

Browse Source
This commit is contained in:
divious1
2020-05-26 23:31:52 -04:00
parent 12e3ef07de 7ac6719584
commit 0357c1663f
827 changed files with 29146 additions and 61189 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.circleci/config.yml
+16 -13
View File
@@ -144,7 +144,7 @@ jobs:
python bin/generate.py --path . --output package -v
# make a copy of use_case_lib in order to have ES work :-(
cp package/default/use_case_library.conf package/default/analyticstories.conf
- run:
- run:
name: copy lookups .csv files
command: |
cd security-content
@@ -261,17 +261,19 @@ jobs:
- run: *apt-install
- aws-cli/setup:
profile-name: default
- run:
- run:
name: update community api sources
command: |
cd security-content
aws s3 cp stories s3://security-content-community/stories --recursive --exclude "*" --include "*.yml"
aws s3 cp baselines s3://security-content-community/baselines --recursive --exclude "*" --include "*.yml"
aws s3 cp detections s3://security-content-community/detections --recursive --exclude "*" --include "*.yml"
aws s3 cp investigations s3://security-content-community/investigations --recursive --exclude "*" --include "*.yml"
aws s3 cp responses s3://security-content-community/responses --recursive --exclude "*" --include "*.yml"
aws s3 cp lookups s3://security-content-community/lookups --recursive --exclude "*" --include "*.yml"
aws s3 cp macros s3://security-content-community/macros --recursive --exclude "*" --include "*.yml"
aws s3 cp stories s3://security-content/stories --recursive --exclude "*" --include "*.yml"
aws s3 cp baselines s3://security-content/baselines --recursive --exclude "*" --include "*.yml"
aws s3 cp detections s3://security-content/detections --recursive --exclude "*" --include "*.yml"
aws s3 cp response_tasks s3://security-content/response_tasks --recursive --exclude "*" --include "*.yml"
aws s3 cp responses s3://security-content/responses --recursive --exclude "*" --include "*.yml"
aws s3 cp lookups s3://security-content/lookups --recursive --exclude "*" --include "*.yml"
aws s3 cp lookups s3://security-content/lookups --recursive --exclude "*" --include "*.csv"
aws s3 cp macros s3://security-content/macros --recursive --exclude "*" --include "*.yml"
aws s3 cp deployments s3://security-content/deployments --recursive --exclude "*" --include "*.yml"
- persist_to_workspace:
root: ~/
paths:
@@ -351,7 +353,7 @@ jobs:
- run:
name: sync latest ESCU to the Attack Range S3 bucket for apps
command: |
aws s3 cp ~/DA-ESS-ContentUpdate-latest.tar.gz s3://attack-range-appbinaries/
aws s3 cp ~/DA-ESS-ContentUpdate-latest.tar.gz s3://attack-range-appbinaries/
# make the file public since it is not by default
aws s3api put-object-acl --bucket attack-range-appbinaries --key DA-ESS-ContentUpdate-latest.tar.gz --acl public-read
master-api-update:
@@ -368,18 +370,19 @@ jobs:
- run: *apt-install
- aws-cli/setup:
profile-name: default
- run:
- run:
name: update api sources
command: |
cd security-content
aws s3 cp stories s3://security-content/stories --recursive --exclude "*" --include "*.yml"
aws s3 cp baselines s3://security-content/baselines --recursive --exclude "*" --include "*.yml"
aws s3 cp detections s3://security-content/detections --recursive --exclude "*" --include "*.yml"
aws s3 cp investigations s3://security-content/investigations --recursive --exclude "*" --include "*.yml"
aws s3 cp response_tasks s3://security-content/response_tasks --recursive --exclude "*" --include "*.yml"
aws s3 cp responses s3://security-content/responses --recursive --exclude "*" --include "*.yml"
aws s3 cp lookups s3://security-content/lookups --recursive --exclude "*" --include "*.yml"
aws s3 cp lookups s3://security-content/lookups --recursive --exclude "*" --include "*.csv"
aws s3 cp macros s3://security-content/macros --recursive --exclude "*" --include "*.yml"
aws s3 cp deployments s3://security-content/deployments --recursive --exclude "*" --include "*.yml"
workflows:
version: 2.1
validate-and-build:
.pre-commit-config.yaml
-3
View File
@@ -9,7 +9,4 @@ repos:
- id: check-yaml
- id: pretty-format-json
args: [--autofix]
- id: flake8
args: [--max-line-length=131]
exclude: 'package/bin/da_ess_contentupdate/|package/bin/splunklib/|venv/|package/bin/escu_contextualize.py|package/bin/escu_investigate.py|package/bin/runstory.py|package/bin/detect.py|package/bin/investigate.py'
- id: requirements-txt-fixer
README.md
+23 -88
View File
@@ -1,8 +1,8 @@
# Splunk Security Content
![security-content](docs/static/logo.png)
![security-content](docs/static/logo.png)
=====
| branch | build status |
@@ -10,81 +10,60 @@
| develop| [![develop status](https://circleci.com/gh/splunk/security-content/tree/develop.svg?style=svg&circle-token=67ad1fa7779c57d7e5bcfc42bd617baf607ec269)](https://circleci.com/gh/splunk/security-content/tree/develop)|
| master | [![master status](https://circleci.com/gh/splunk/security-content/tree/master.svg?style=svg&circle-token=67ad1fa7779c57d7e5bcfc42bd617baf607ec269)](https://circleci.com/gh/splunk/security-content/tree/master)|
Welcome to the Splunk Security Research Team's Security Content Exchange!
This project gives you access to our repository of Analytic Stories--themed security guides that contain that provide background on TTPs, mapped to the MITRE framework, the Lockheed Martin Kill Chain, and CIS controls. They include Splunk searches, machine-learning algorithms, and Splunk Phantom playbooks (where available)—all designed to work together to detect, investigate, and respond to threats.
While this content is available via Splunk Enterprise Security and Enterprise Security Content Updates (https://splunkbase.splunk.com/app/3449/), we have now made it available as an open-source project (which you just found--hi!). The Security Research Content Exchange was designed to bring the community together to improve our collective defenses. By sharing research and analytics, we can help the entire industry craft more effective strategies. This project provides a mechanism to facilitate this exchange.
Welcome to the Splunk Security Content
This project gives you access to our repository of Analytic Stories that are security guides which provide background on TTPs, mapped to the MITRE framework, the Lockheed Martin Kill Chain, and CIS controls. They include Splunk searches, machine-learning algorithms, and Splunk Phantom playbooks (where available)—all designed to work together to detect, investigate, and respond to threats.
# Usage
The Splunk Security Content Exchange can be used via:
The Splunk Security Content can be used via:
#### [Splunk App](https://github.com/splunk/security-content/releases)
Grab the latest release of DA-ESS-ContentUpdate and install it on a Splunk Enterprise server (search head).
Grab the latest release of DA-ESS-ContentUpdate and install it on a Splunk Enterprise instance.
#### [API](https://docs.splunkresearch.com/?version=latest)
```
curl -s https://content.splunkresearch.com | jq
{
"hello": "welcome to Splunks Research security content api",
"available_endpoints": [
"/stories",
"/detections",
"/investigations",
"/baselines",
"/responses",
"/package"
]
"hello": "welcome to Splunks Research security content api"
}
```
# How to Get Started
While you you can configure each detection individually, the real power lies in running the Analytic Stories end-to-end. Using the Analytic Story Execution (ASX) app (https://github.com/splunk/analytic_story_execution), you can funnel the results to investigations, reducing the need for manual intervention. When available, you can automatically trigger reponses, as well. The alerts you'll get include context (history, correlations, etc.), so they help you better understand their importance. The net effect is a more efficient workflow, as well as more comprehensive, effective defenses.
Follow the instructions below to get started.
#### [GitHub Workflow](https://github.com/splunk/security-content/wiki/2.-Installation-and-Usage)
Create your customized version of Security Content by forking this project and following this guide.
# What's in an Analytic Story?
[Analytic Stories](https://github.com/splunk/security-content/blob/develop/docs/stories_categories.md) and their corresponding searches are composed of **.yml** files (manifests) and associated .conf files. The stories reside in [/stories](https://github.com/splunk/security-content/tree/develop/stories) and the searches live in [/detections](https://github.com/splunk/security-content/tree/develop/detections).
[Analytic Stories](https://github.com/splunk/security-content/blob/develop/docs/stories_categories.md) and their corresponding searches are composed of **.yml** files (manifests) and associated .conf files. The stories reside in [/stories](https://github.com/splunk/security-content/tree/develop/stories) and the searches live in [/detections](https://github.com/splunk/security-content/tree/develop/detections).
Manifests contain a number of mandatory and optional fields. You can see the full field list for each piece of content [here](https://github.com/splunk/security-content/tree/develop/docs#spec-documentation).
# Customize to your Environment
After release [1.0.46](https://github.com/splunk/security-content/releases) we introduced a concept of input(pre-filter) and output(post-filter) macros for each of our detection search. The intention behind introducing these macros is primarily to help our users to update the macro definition “once” and those changes will be applicable across all detections that leverage that macro and local to your Splunk Environment.
After release [1.0.46](https://github.com/splunk/security-content/releases) we introduced a concept of **input(pre-filter)** and **output(post-filter)** macros for each of our detection search. The intention behind introducing these macros is primarily to help our users to update the macro definition “once” and those changes will be applicable across all detections that leverage that macro and local to your Splunk Environment.
**input(pre-filter):** This macro is to specify your environment-specific configurations (index, source, sourcetype, etc.) to get the specific data sources that you would like to bring in. Replace the macro definition with configurations for your Splunk environment.
**input(pre-filter):** This macro is to specify your environment-specific configurations (index, source, sourcetype, etc.) to get the specific data sources that you would like to bring in. Replace the macro definition with configurations for your Splunk environment. For example the [sysmon](macros/sysmon.yml) **input macro** can be modified to the local splunk deployments index or sourcetype.
**output(post-filter):** This macro is to specify your environment-specific values (eg: dest, user), to filter out known false positives.. Replace the macro definition with values that you’d like to exclude from detection results. Think of this as a whitelisting/blacklisting using macros.
**output(post-filter):** This macro is to specify your environment-specific values (eg: dest, user), to filter out known false positives.. Replace the macro definition with values that you’d like to exclude from detection results. Think of this as a whitelisting/blacklisting using macros. A good example
Note: we are currently working on coming up with a better naming convention and making this consistent across all our detections, investigations and baselines. Suggestions are welcomed :stuck_out_tongue:
# Execute an Analytic Story
Download and install the latest version of [Splunk Analytic Story Execution]
Download and install the latest version of [Splunk Analytic Story Execution]
(https://github.com/splunk/analytic_story_execution/releases). This Splunk application will help the user do the following:
1. Execute an analytic story in an adhoc mode and view the results.
2. Schedule all the detection searches in an analytic story.
3. Update security-content via an API
# Writing Content
Before you begin, follow the steps to install **dependencies and pre-commit hooks** under ["Developing"](https://github.com/splunk/security-content#developing).
Before you begin, follow the steps to install **dependencies and pre-commit hooks** under ["Developing"](https://github.com/splunk/security-content#developing).
1. Select the content [piece](https://github.com/splunk/security-content#content-parts) you want to write.
2. Copy an example and edit it to suit your needs. At a minimum, you must write a [story](stories/), [a detection search](detections/), and an [investigative search](investigations/).
3. Make a pull request. The pull request will trigger CircleCI, a continuous-integration app thatintegrates with a VCS and automatically runs a series of steps every time that it detects a change to your repository. A CircleCI build consists of a series of steps, usually Dependencies, Testing, and Deployment. If your tests pass, you're good to go! If the CircleCI check fails, refer to [troubleshooting](https://github.com/splunk/security-content#troubleshooting).
For a more detailed explanation on how to contribute to the project, please see ["Contributing"](#Contributing)
# Security Content Layout
![](docs/static/structure.png)
# Security Content
#### Content Parts
* [stories/](stories/): All Analytic Stories
* [stories/](stories/): All Analytic Stories
* [detections/](detections/): Splunk Enterprise, Splunk UBA, and Splunk Phantom detections that power Analytic Stories
* [investigations/](investigations/): Splunk Enterprise and Splunk Phantom investigative searches and playbooks employed by Analytic Stories
* [response_tasks/](response_tasks/): Splunk Enterprise and Splunk Phantom investigative searches and playbooks employed by Analytic Stories
* [responses/](responses/): Automated Splunk Enterprise and Splunk Phantom responses triggered by Analytic Stories
* [baselines/](baselines/): Splunk Phantom and Splunk Enterprise baseline searches needed to support detection searches in Analytic Stories
@@ -92,46 +71,8 @@ For a more detailed explanation on how to contribute to the project, please see
* [package/](package/): Splunk content app-source files, including lookups, binaries, and default config files
* [bin/](bin/): All binaries required to produce and test content
# Docs
* [docs/](docs/): Documentation for all spec files
* [spec/](spec/): All spec files that describe the security content
# Developing
##### Dependecies and Pre-Commit Hooks
Install project dependecies and tests that run before content is committed:
1. Create virtualenv and install requirements: `virtualenv venv && source venv/bin/activate && pip install -r requirements.txt`.
2. Install `pre-commit install`.
##### CI Tools
Tools that help with testing CI jobs:
1. Install CircleCI [CLI Tool](https://circleci.com/docs/2.0/local-cli/).
2. To test a local change to CircleCI or build, make sure you are running Docker, then enter
`circleci local execute -e GITHUB_TOKEN=$GITHUB_TOKEN --branch <your branch>`.
##### Generate Docs from Schema
To automatically generate docs from schema:
1. Install https://github.com/adobe/jsonschema2md.
2. Enter `jsonschema2md -d spec/v2/detections.spec.json -o docs`.
# Troubleshooting
### Our Automated Tests
1. [CI](https://github.com/splunk/security-content/blob/44946063173f7bc9921f0da0aa62139c084d1c51/.circleci/config.yml#L27) validates that the content was written to spec using [`validate.py`](https://github.com/splunk/security-content/blob/runstory/bin/generate.py). To run validation manually, run: `python bin/validate.py --path . --verbose`.
2. [CI](https://github.com/splunk/security-content/blob/44946063173f7bc9921f0da0aa62139c084d1c51/.circleci/config.yml#L60) generates Splunk configuration files using [`generate.py`](https://github.com/splunk/security-content/blob/develop/bin/generate.py). If you want to export Splunk .conf files manually from the content, run `python bin/generate.py --path . --output package --verbose`.
3. [CI](https://github.com/splunk/security-content/blob/44946063173f7bc9921f0da0aa62139c084d1c51/.circleci/config.yml#L107) builds a DA-ESS-ContentUpdate Splunk package using the [Splunk Packaging Toolkit](http://dev.splunk.com/view/packaging-toolkit/SP-CAAAE9V).
4. [CI](https://github.com/splunk/security-content/blob/44946063173f7bc9921f0da0aa62139c084d1c51/.circleci/config.yml#L145) tests the newly produced package using [Splunk Appinspect](http://dev.splunk.com/view/appinspect/SP-CAAAE9U).
* note that [requirements.txt](https://github.com/splunk/security-content/blob/develop/requirements.txt) hard codes the versions for packages we use [dependabot](https://dependabot.com/) to make sure we safely always upgrade to the latest versions.
## Customize to Your Environment
Release 1.0.46 introduced input(pre-filter) and output(post-filter) macros for each of our detection searches. These macros let you update a macro definition once and then apply the new definition across all detections that leverage that macro. These changes will be local to your Splunk environment.
input(pre-filter): This macro specifies your environment-specific configurations (index, source, sourcetype, etc.) to get the specific data sources that you require. Replace the macro definition with configurations for your Splunk environment.
output(post-filter): This macro specifies your environment-specific values (dest, user, etc,), to filter out known false positives. Replace the macro definition with values that you'd like to exclude from detection results. Think of this as whitelisting/blacklisting using macros.
Note: Coming soon is an improved naming convention that will be consistent across all of our detections, investigations, and baselines.
# Contribution
We welcome feedback and contributions from the community! Please see our [contribution guidelines](docs/CONTRIBUTING.md) for more information on how to get involved.
## Support
Please use the [GitHub Issue Tracker](https://github.com/splunk/security-content/issues) to submit bugs or request features.
@@ -141,9 +82,3 @@ If you have questions or need support, you can:
* Post a question to [Splunk Answers](http://answers.splunk.com)
* Join the [#security-research](https://splunk-usergroups.slack.com/messages/C1RH09ERM/) room in the [Splunk Slack channel](http://splunk-usergroups.slack.com)
* If you are a Splunk Enterprise customer with a valid support entitlement contract and have a Splunk-related question, you can also open a support case on the https://www.splunk.com/ support portal
## Contributing
We welcome feedback and contributions from the community! Please see our [contribution guidelines](docs/CONTRIBUTING.md) for more information on how to get involved.
# To Dos
* Build CLI for interacting and developing
baselines/add_prohibited_processes_to_enterprise_security.yml
+20
View File
@@ -0,0 +1,20 @@
name: Add Prohibited Processes to Enterprise Security
id: 251930a5-1451-4428-bb13-eed5775be0ce
version: 1
date: '2017-09-15'
description: This search takes the existing interesting process table from ES, filters
out any existing additions added by ESCU and then updates the table with processes
identified by ESCU that should be prohibited on your endpoints.
how_to_implement: This search should be run on each new install of ESCU.
author: David Dorsey, Splunk
search: '| inputlookup interesting_processes_lookup | search note!=ESCU* | inputlookup
append=T prohibitedProcesses_lookup | fillnull value=* dest dest_pci_domain | fillnull
value=false is_required is_secure | fillnull value=true is_prohibited | outputlookup
interesting_processes_lookup | stats count'
tags:
analytics_story:
- 'Emotet Malware DHS Report TA18-201A '
- Monitor for Unauthorized Software
- SamSam Ransomware
detections:
- Prohibited Software On Endpoint
baselines/add_prohibited_processes_to_es.yml
-43
View File
@@ -1,43 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| inputlookup interesting_processes_lookup | search note!=ESCU* | inputlookup
append=T prohibitedProcesses_lookup | fillnull value=* dest dest_pci_domain
| fillnull value=false is_required is_secure | fillnull value=true is_prohibited
| outputlookup interesting_processes_lookup | stats count'
creation_date: '2017-06-27'
data_metadata:
data_models: []
data_source:
- Splunk Enterprise
providing_technologies:
- Splunk Enterprise Security
description: This search takes the existing interesting process table from ES, filters
out any existing additions added by ESCU and then updates the table with processes
identified by ESCU that should be prohibited on your endpoints.
eli5: This search outputs the interesting processes lookup table and filters out all
processes in the table that haven't already been inserted by ESCU. It then appends
to those results all the processes currently identified by ESCU that should be prohibited.
Next, it fills in the required fields with processes identified by ESCU, and then
writes the results back to the interesting process lookup table. This is done so
any new processes identified that should be prohibited will be added to the lookup
table without creating any duplicate entries.
how_to_implement: This search should be run on each new install of ESCU.
id: 251930a5-1451-4428-bb13-eed5775be0ce
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2017-09-15'
name: Add Prohibited Processes to Enterprise Security
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/approved_aws_service_accounts.yml
-45
View File
@@ -1,45 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail errorCode=success | rename userName as identity
| search NOT [inputlookup identity_lookup_expanded | fields identity] | stats
count by identity | table identity | outputlookup aws_service_accounts | stats
count
creation_date: '2018-03-12'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search looks for successful API activity in CloudTrail within the
last 30 days, filters out known users from the identity table, and outputs values
of users into `aws_service_accounts.csv` lookup file.
eli5: We first look for all successful CloudTrail API activity caused by types of
user accounts and then remove all the events caused by users in the Identity table.
This generates a list of accounts--typically service accounts--configured in your
AWS environment. We output this list of service accounts to `aws_service_accounts.csv`.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Please validate the service account entires in `aws_service_accounts.csv`,
which is a lookup file created as a result of running this support search. Please
remove the entries of service accounts that are not legitimate.
id: fc0edc95-ff2b-48b1-5f6f-63ga3789fd43
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2018-12-03'
name: Create a list of approved AWS service accounts
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '2.0'
baselines/assets_sorted_by_category.yml
-40
View File
@@ -1,40 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| from datamodel Identity_Management.All_Assets | stats count values(nt_host)
by category | sort -count'
creation_date: '2017-06-11'
data_metadata:
data_models:
- Identity_Management
data_source:
- Splunk Enterprise Security
providing_technologies:
- Splunk Enterprise Security
description: This search shows you every asset category you have and the assets that
belong to those categories.
eli5: This search gives you the number and the names of the hosts of each host in
your environment by category. It will then sort them by the count.
how_to_implement: To successfully implement this search you must first leverage the
Assets and Identity framework in Enterprise Security to populate your assets_by_str.csv
file which should then be mapped to the Identity_Management data model. The Identity_Management
data model will contain a list of known authorized company assets. Ensure that all
inventoried systems are constantly vetted and updated.
id: dcfd6b40-42f9-469d-a433-2e53f7489ff9
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2017-09-13'
name: Count of assets by category
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '1.0'
baselines/baseline_api_calls_per_user_arn.yml
-47
View File
@@ -1,47 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventType=AwsApiCall | spath output=arn path=userIdentity.arn
| bucket _time span=1h | stats count as apiCalls by _time, arn | stats count(apiCalls)
as numDataPoints, latest(apiCalls) as latestCount, avg(apiCalls) as avgApiCalls,
stdev(apiCalls) as stdevApiCalls by arn | table arn, latestCount, numDataPoints,
avgApiCalls, stdevApiCalls | outputlookup api_call_by_user_baseline | stats
count
creation_date: '2018-04-09'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search establishes, on a per-hour basis, the average and the standard
deviation of the number of API calls made by each user. Also recorded is the number
of data points for each user. This table is then outputted to a lookup file to allow
the detection search to operate quickly.
eli5: This search returns all log events that are API calls, pulls out the ARN that
initiated each call, and collects them in one-hour groupings. Next, it calculates
the number of API calls made per ARN per hour. For each ARN, it calculates the average
and standard deviation of this count on a per-hour basis. It also includes the
number of data points each ARN had. This table is then stored in a lookup file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
id: fc0edc96-ff2b-48b0-9f6f-63da3783fd63
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-04-09'
name: Baseline of API Calls per User ARN
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/baseline_aws_regions.yml
-45
View File
@@ -1,45 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail StartInstances | stats earliest(_time) as earliest
latest(_time) as latest by awsRegion | outputlookup previously_seen_aws_regions.csv
| stats count
creation_date: '2018-01-08'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search looks for CloudTrail events where an AWS instance is started
and creates a baseline of most recent time (latest) and the first time (earliest)
we've seen this region in our dataset grouped by the value awsRegion for the last
30 days
eli5: In this support search, we create a table of the first time (earliest) and most
recent time (latest) that this region has been seen in our dataset, grouped by the
value `awsRegion`. We only look for those events where an instance has been started.
All of these entries will be added to the `previously_seen_aws_regions.csv` lookup
file, which will act like a baseline for detections. Please validate the entries
of region names in the lookup file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
id: fc0edc95-ff2b-48b0-9f6f-63da3789fd63
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2018-01-08'
name: Previously Seen AWS Regions
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '1.0'
baselines/baseline_blocked_vpc_traffic.yml
-54
View File
@@ -1,54 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudwatchlogs:vpcflow action=blocked (src_ip=10.0.0.0/8
OR src_ip=172.16.0.0/12 OR src_ip=192.168.0.0/16) ( dest_ip!=10.0.0.0/8 AND
dest_ip!=172.16.0.0/12 AND dest_ip!=192.168.0.0/16) | bucket _time span=1h |
stats count as numberOfBlockedConnections by _time, src_ip | stats count(numberOfBlockedConnections)
as numDataPoints, latest(numberOfBlockedConnections) as latestCount, avg(numberOfBlockedConnections)
as avgBlockedConnections, stdev(numberOfBlockedConnections) as stdevBlockedConnections
by src_ip | table src_ip, latestCount, numDataPoints, avgBlockedConnections,
stdevBlockedConnections | outputlookup baseline_blocked_outbound_connections
| stats count
creation_date: '2018-04-26'
data_metadata:
data_source:
- AWS VPC Flow Logs
data_sourcetypes:
- aws:cloudwatchlogs:vpcflow
providing_technologies:
- AWS
description: This search establishes, on a per-hour basis, the average and the standard
deviation of the number of outbound connections blocked in your VPC flow logs by
each source IP address (IP address of your EC2 instances). Also recorded is the
number of data points for each source IP. This table outputs to a lookup file to
allow the detection search to operate quickly.
eli5: Use this search to create a baseline of blocked outbound network connections
by each source IP in your AWS environment. This search returns all log events that
correspond to a blocked outbound network connection, extracts the source IP from
where the outbound connection was initiated, and collects the events in one-hour
groupings. Next, it calculates the number of outbound connections blocked per hour.
For each source IP, it calculates the average and standard deviation of this count
on a per-hour basis. It also includes the number of data points each source IP
had. This table is then stored in a lookup file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your `VPC flow
logs.`.
id: fc0edd96-ff2b-48b0-9f1f-63da3782fd63
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2018-05-07'
name: Baseline of blocked outbound traffic from AWS
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '1.0'
baselines/baseline_cloud_compute_regions.yml
-50
View File
@@ -1,50 +0,0 @@
baseline:
splunk:
lookups:
- previously_seen_cloud_regions
macros:
- previously_seen_cloud_regions_input_filter
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| tstats earliest(_time) as firstTime, latest(_time) as lastTime from
datamodel=Cloud_Infrastructure.Compute where Compute.action=start `previously_seen_cloud_regions_input_filter`
by Compute.region | `drop_dm_object_name("Compute")` | outputlookup previously_seen_cloud_regions
| stats count'
creation_date: '2019-10-02'
data_metadata:
data_models:
- Cloud_Infrastructure
data_source:
- Cloud Infrastructure Logs
providing_technologies:
- AWS
- Azure
- GCP
description: This search looks for cloud compute events where a compute instance is
started and creates a baseline of most recent time, `lastTime` and the first time
`firstTime` we've seen this region in our dataset grouped by the region for the
last 30 days
eli5: In this support search, we create a table of the first time `firstTime` and
most recent time `lastTime` that this region has been seen in our dataset, grouped
by the region. We only look for those events where an instance has been started.
All of these entries will be added to the `previously_seen_cloud_regions` lookup
file, which will act like a baseline for detections.
how_to_implement: You must be ingesting the approrpiate cloud infrastructure logs
and have the Security Research cloud data model installed.
id: b5e232db-dec6-4db8-aaa1-dd5474521e40
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2019-10-02'
name: Previously Seen Cloud Regions
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/baseline_network_acl_modifications.yml
-49
View File
@@ -1,49 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail `network_acl_events` | spath output=arn path=userIdentity.arn
| bucket _time span=1h | stats count as apiCalls by _time, arn | stats count(apiCalls)
as numDataPoints, latest(apiCalls) as latestCount, avg(apiCalls) as avgApiCalls,
stdev(apiCalls) as stdevApiCalls by arn | table arn, latestCount, numDataPoints,
avgApiCalls, stdevApiCalls | outputlookup network_acl_activity_baseline | stats
count
creation_date: '2018-05-21'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search establishes, on a per-hour basis, the average and the standard
deviation of the number of API calls that were related to network ACLs made by each
user. Also recorded is the number of data points for each user. This table is then
outputted to a lookup file to allow the detection search to operate quickly.
eli5: Use this search to create a baseline for API calls related to network ACLs for
the users who initiated this activity. It returns all logged API calls for network
activity, pulls out the ARN that initiated each call, and collects the `eventNames`
in one-hour groupings. Next, it calculates the number of API calls made per ARN
per-hour. For each ARN, it calculates the average and standard deviation of this
count on a per-hour basis. It also includes the number of data points for each ARN.
This table is stored in a lookup file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs. To add or remove API event names for network ACLs, edit the macro `network_acl_events`.
id: fc0edd96-ff2b-4810-9f1f-63da3783fd63
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2018-05-21'
name: Baseline of Network ACL Activity by ARN
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '1.0'
baselines/baseline_of_api_calls_per_user_arn.yml
+22
View File
@@ -0,0 +1,22 @@
name: Baseline of API Calls per User ARN
id: fc0edc96-ff2b-48b0-9f6f-63da3783fd63
version: 1
date: '2018-04-09'
description: This search establishes, on a per-hour basis, the average and the standard
deviation of the number of API calls made by each user. Also recorded is the number
of data points for each user. This table is then outputted to a lookup file to allow
the detection search to operate quickly.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
author: David Dorsey, Splunk
search: '`cloudtrail` eventType=AwsApiCall | spath output=arn path=userIdentity.arn
| bucket _time span=1h | stats count as apiCalls by _time, arn | stats count(apiCalls)
as numDataPoints, latest(apiCalls) as latestCount, avg(apiCalls) as avgApiCalls,
stdev(apiCalls) as stdevApiCalls by arn | table arn, latestCount, numDataPoints,
avgApiCalls, stdevApiCalls | outputlookup api_call_by_user_baseline | stats count'
tags:
analytics_story:
- AWS User Monitoring
detections:
- Detect Spike in AWS API Activity
baselines/baseline_of_blocked_outbound_traffic_from_aws.yml
+28
View File
@@ -0,0 +1,28 @@
name: Baseline of blocked outbound traffic from AWS
id: fc0edd96-ff2b-48b0-9f1f-63da3782fd63
version: 1
date: '2018-05-07'
description: This search establishes, on a per-hour basis, the average and the standard
deviation of the number of outbound connections blocked in your VPC flow logs by
each source IP address (IP address of your EC2 instances). Also recorded is the
number of data points for each source IP. This table outputs to a lookup file to
allow the detection search to operate quickly.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your `VPC flow
logs.`.
author: Bhavin Patel, Splunk
search: '`cloudwatchlogs_vpcflow` action=blocked (src_ip=10.0.0.0/8 OR src_ip=172.16.0.0/12
OR src_ip=192.168.0.0/16) ( dest_ip!=10.0.0.0/8 AND dest_ip!=172.16.0.0/12 AND dest_ip!=192.168.0.0/16)
| bucket _time span=1h | stats count as numberOfBlockedConnections by _time, src_ip
| stats count(numberOfBlockedConnections) as numDataPoints, latest(numberOfBlockedConnections)
as latestCount, avg(numberOfBlockedConnections) as avgBlockedConnections, stdev(numberOfBlockedConnections)
as stdevBlockedConnections by src_ip | table src_ip, latestCount, numDataPoints,
avgBlockedConnections, stdevBlockedConnections | outputlookup baseline_blocked_outbound_connections
| stats count'
tags:
analytics_story:
- AWS Network ACL Activity
- Command and Control
- Suspicious AWS Traffic
detections:
- Detect Spike in blocked Outbound Traffic from your AWS
baselines/train_unusually_long_commandlines.yml → baselines/baseline_of_command_line_length___mltk.yml
+20 -41
View File
@@ -1,34 +1,12 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -31d@d
latest_time: -1d@d
search: '| tstats `security_content_summariesonly` count min(_time) as start_time max(_time) as
end_time FROM datamodel=Endpoint.Processes by Processes.user Processes.dest
Processes.process_name Processes.process | `drop_dm_object_name(Processes)`
| search user!=unknown | `security_content_ctime(start_time)`| `security_content_ctime(end_time)`| eval processlen=len(process)
| fit DensityFunction processlen by user into cmdline_pdfmodel'
creation_date: '2019-05-08'
data_metadata:
data_models:
- Endpoint
data_source:
- Endpoint Intel
providing_technologies:
- Carbon Black Response
- CrowdStrike Falcon
- Sysmon
- Tanium
- Ziften
name: Baseline of Command Line Length - MLTK
id: d2a4d85b-fc6a-47a0-82f6-bc1ec2ebc459
version: 1
date: '2019-05-08'
description: This search is used to build a Machine Learning Toolkit (MLTK) model
to characterize the length of the command lines observed for each user in the environment.
By default, the search uses the last 30 days of data to build the model. The model
created by this search is then used in the corresponding detection search, which
identifies outliers in the length of the command line.
eli5: Create a machine-learning (ML) model to characterize the length of the command
lines used in your environment. This can help you identify unusually long ones that
may indicate that attackers are executing commands on yout systems.
how_to_implement: You must be ingesting endpoint data and populating the Endpoint
data model. In addition, you must have the Machine Learning Toolkit (MLTK) version
>= 4.2 installed, along with any required dependencies. Depending on the number
@@ -39,18 +17,19 @@ how_to_implement: You must be ingesting endpoint data and populating the Endpoin
a longer period of time, which may give you better results. You may also want to
periodically re-run this search to rebuild the model with the latest data. More
information on the algorithm used in the search can be found at `https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms#DensityFunction`.
id: d2a4d85b-fc6a-47a0-82f6-bc1ec2ebc459
known_false_positives: ''
maintainers:
- company: Splunk
email: rvaldez@splunk.com
name: Rico Valdez
modification_date: '2019-05-08'
name: Baseline of Command Line Length - MLTK
original_authors:
- company: Splunk
email: rvaldez@splunk.com
name: Rico Valdez
spec_version: 2
type: splunk
version: '1.0'
author: Rico Valdez, Splunk
search: '| tstats `security_content_summariesonly` count min(_time) as start_time
max(_time) as end_time FROM datamodel=Endpoint.Processes by Processes.user Processes.dest
Processes.process_name Processes.process | `drop_dm_object_name(Processes)` | search
user!=unknown | `security_content_ctime(start_time)`| `security_content_ctime(end_time)`|
eval processlen=len(process) | fit DensityFunction processlen by user into cmdline_pdfmodel'
tags:
analytics_story:
- Possible Backdoor Activity Associated With MUDCARP Espionage Campaigns
- Ransomware
- Suspicious Command-Line Executions
- Suspicious MSHTA Activity
- Unusual Processes
detections:
- Detect Prohibited Applications Spawning cmd.exe
- Unusually Long Command Line - MLTK
baselines/train_dns_query_length.yml → baselines/baseline_of_dns_query_length___mltk.yml
+16 -38
View File
@@ -1,31 +1,12 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -31d@d
latest_time: -1d@d
search: '| tstats `security_content_summariesonly` count from datamodel=Network_Resolution by DNS.query
DNS.record_type | search DNS.record_type=* | `drop_dm_object_name("DNS")` |
eval query_length = len(query) | fit DensityFunction query_length by record_type
into dns_query_pdfmodel'
creation_date: '2019-05-08'
data_metadata:
data_models:
- Network_Resolution
data_source:
- DNS
providing_technologies:
- Splunk Stream
- Bro
name: Baseline of DNS Query Length - MLTK
id: c914844c-0ff5-4efc-8d44-c063443129ba
version: 1
date: '2019-05-08'
description: This search is used to build a Machine Learning Toolkit (MLTK) model
to characterize the length of the DNS queries for each DNS record type observed
in the environment. By default, the search uses the last 30 days of data to build
the model. The model created by this search is then used in the corresponding detection
search, which uses it to identify outliers in the length of the DNS query.
eli5: Create a machine-learning (ML) model to characterize the length of DNS requests
seen in your environment to help identify unusually long ones that may be indicative
of attacker infrastrucutre or the use of DNS as a command-and-control channel in
your environment.
how_to_implement: To successfully implement this search, you will need to ensure that
DNS data is populating the Network_Resolution data model. In addition, you must
have the Machine Learning Toolkit (MLTK) version >= 4.2 installed, along with any
@@ -34,18 +15,15 @@ how_to_implement: To successfully implement this search, you will need to ensure
period of time, which may give you better results. You may also want to periodically
re-run this search to rebuild the model with the latest data. More information on
the algorithm used in the search can be found at `https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms#DensityFunction`.
id: c914844c-0ff5-4efc-8d44-c063443129ba
known_false_positives: ''
maintainers:
- company: Splunk
email: rvaldez@splunk.com
name: Rico Valdez
modification_date: '2019-05-08'
name: Baseline of DNS Query Length - MLTK
original_authors:
- company: Splunk
email: rvaldez@splunk.com
name: Rico Valdez
spec_version: 2
type: splunk
version: '1.0'
author: Rico Valdez, Splunk
search: '| tstats `security_content_summariesonly` count from datamodel=Network_Resolution
by DNS.query DNS.record_type | search DNS.record_type=* | `drop_dm_object_name("DNS")`
| eval query_length = len(query) | fit DensityFunction query_length by record_type
into dns_query_pdfmodel'
tags:
analytics_story:
- Command and Control
- Hidden Cobra Malware
- Suspicious DNS Traffic
detections:
- DNS Query Length Outliers - MLTK
baselines/baseline_of_excessive_aws_instances_launched_by_user___mltk.yml
+33
View File
@@ -0,0 +1,33 @@
name: Baseline of Excessive AWS Instances Launched by User - MLTK
id: fa5634df-fb05-4b4b-aba0-6115138bb1ba
version: 1
date: '2019-11-14'
description: This search is used to build a Machine Learning Toolkit (MLTK) model
for how many RunInstances users do in the environment. By default, the search uses
the last 90 days of data to build the model. The model created by this search is
then used in the corresponding detection search, which identifies subsequent outliers
in the number of RunInstances performed by a user in a small time window.
how_to_implement: 'You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs.\
In addition, you must have the Machine Learning Toolkit (MLTK) version >= 4.2 installed,
along with any required dependencies. Depending on the number of users in your environment,
you may also need to adjust the value for max_inputs in the MLTK settings for the
DensityFunction algorithm, then ensure that the search completes in a reasonable
timeframe. By default, the search builds the model using the past 30 days of data.
You can modify the search window to build the model over a longer period of time,
which may give you better results. You may also want to periodically re-run this
search to rebuild the model with the latest data.\
More information on the algorithm used in the search can be found at `https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms#DensityFunction`.'
author: Jason Brewer, Splunk
search: '`cloudtrail` eventName=RunInstances errorCode=success `ec2_excessive_runinstances_mltk_input_filter`
| bucket span=10m _time | stats count as instances_launched by _time src_user |
fit DensityFunction instances_launched threshold=0.0005 into ec2_excessive_runinstances_v1'
tags:
analytics_story:
- Cloud Cryptomining
- Suspicious AWS EC2 Activities
detections:
- Abnormally High AWS Instances Launched by User - MLTK
baselines/baseline_of_excessive_aws_instances_terminated_by_user___mltk.yml
+33
View File
@@ -0,0 +1,33 @@
name: Baseline of Excessive AWS Instances Terminated by User - MLTK
id: b28ed6de-e4ba-40f7-ae0a-93a088c774ab
version: 1
date: '2019-11-14'
description: This search is used to build a Machine Learning Toolkit (MLTK) model
for how many TerminateInstances users do in the environment. By default, the search
uses the last 90 days of data to build the model. The model created by this search
is then used in the corresponding detection search, which identifies subsequent
outliers in the number of TerminateInstances performed by a user in a small time
window.
how_to_implement: 'You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs.\
In addition, you must have the Machine Learning Toolkit (MLTK) version >= 4.2 installed,
along with any required dependencies. Depending on the number of users in your environment,
you may also need to adjust the value for max_inputs in the MLTK settings for the
DensityFunction algorithm, then ensure that the search completes in a reasonable
timeframe. By default, the search builds the model using the past 30 days of data.
You can modify the search window to build the model over a longer period of time,
which may give you better results. You may also want to periodically re-run this
search to rebuild the model with the latest data.\
More information on the algorithm used in the search can be found at `https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms#DensityFunction`.'
author: Jason Brewer, Splunk
search: '`cloudtrail` eventName=TerminateInstances errorCode=success `ec2_excessive_terminateinstances_mltk_input_filter`
| bucket span=10m _time | stats count as instances_terminated by _time src_user
| fit DensityFunction instances_terminated threshold=0.0005 into ec2_excessive_terminateinstances_v1'
tags:
analytics_story:
- Suspicious AWS EC2 Activities
detections:
- Abnormally High AWS Instances Terminated by User - MLTK
baselines/baseline_of_network_acl_activity_by_arn.yml
+23
View File
@@ -0,0 +1,23 @@
name: Baseline of Network ACL Activity by ARN
id: fc0edd96-ff2b-4810-9f1f-63da3783fd63
version: 1
date: '2018-05-21'
description: This search establishes, on a per-hour basis, the average and the standard
deviation of the number of API calls that were related to network ACLs made by each
user. Also recorded is the number of data points for each user. This table is then
outputted to a lookup file to allow the detection search to operate quickly.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs. To add or remove API event names for network ACLs, edit the macro `network_acl_events`.
author: Bhavin Patel, Splunk
search: '`cloudtrail` `network_acl_events` | spath output=arn path=userIdentity.arn
| bucket _time span=1h | stats count as apiCalls by _time, arn | stats count(apiCalls)
as numDataPoints, latest(apiCalls) as latestCount, avg(apiCalls) as avgApiCalls,
stdev(apiCalls) as stdevApiCalls by arn | table arn, latestCount, numDataPoints,
avgApiCalls, stdevApiCalls | outputlookup network_acl_activity_baseline | stats
count'
tags:
analytics_story:
- AWS Network ACL Activity
detections:
- Detect Spike in Network ACL Activity
baselines/baseline_of_s3_bucket_deletion_activity_by_arn.yml
+22
View File
@@ -0,0 +1,22 @@
name: Baseline of S3 Bucket deletion activity by ARN
id: fc0edd96-ff2b-48b0-9f1f-63eq3783fd63
version: 1
date: '2018-07-17'
description: This search establishes, on a per-hour basis, the average and standard
deviation for the number of API calls related to deleting an S3 bucket by each user.
Also recorded is the number of data points for each user. This table is then outputted
to a lookup file to allow the detection search to operate quickly.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
author: Bhavin Patel, Splunk
search: '`cloudtrail` eventName=DeleteBucket | spath output=arn path=userIdentity.arn
| bucket _time span=1h | stats count as apiCalls by _time, arn | stats count(apiCalls)
as numDataPoints, latest(apiCalls) as latestCount, avg(apiCalls) as avgApiCalls,
stdev(apiCalls) as stdevApiCalls by arn | table arn, latestCount, numDataPoints,
avgApiCalls, stdevApiCalls | outputlookup s3_deletion_baseline | stats count'
tags:
analytics_story:
- Suspicious AWS S3 Activities
detections:
- Detect Spike in S3 Bucket deletion
baselines/baseline_of_security_group_activity_by_arn.yml
+23
View File
@@ -0,0 +1,23 @@
name: Baseline of Security Group Activity by ARN
id: fc0edd96-ff2b-48b0-9f1f-63da3783fd63
version: 1
date: '2018-04-17'
description: This search establishes, on a per-hour basis, the average and the standard
deviation for the number of API calls related to security groups made by each user.
Also recorded is the number of data points for each user. This table is then outputted
to a lookup file to allow the detection search to operate quickly.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs. To add or remove API event names for security groups, edit the macro `security_group_api_calls`.
author: Bhavin Patel, Splunk
search: '`cloudtrail` `security_group_api_calls` | spath output=arn path=userIdentity.arn
| bucket _time span=1h | stats count as apiCalls by _time, arn | stats count(apiCalls)
as numDataPoints, latest(apiCalls) as latestCount, avg(apiCalls) as avgApiCalls,
stdev(apiCalls) as stdevApiCalls by arn | table arn, latestCount, numDataPoints,
avgApiCalls, stdevApiCalls | outputlookup security_group_activity_baseline | stats
count'
tags:
analytics_story:
- AWS User Monitoring
detections:
- Detect Spike in Security Group Activity
baselines/train_smb_traffic_spike.yml → baselines/baseline_of_smb_traffic___mltk.yml
+21 -41
View File
@@ -1,35 +1,13 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -31d@d
latest_time: -1d@d
search: '| tstats `security_content_summariesonly` count from datamodel=Network_Traffic where All_Traffic.dest_port=139
OR All_Traffic.dest_port=445 OR All_Traffic.app=smb by _time span=10m, All_Traffic.src
| eval HourOfDay=strftime(_time, "%H") | eval DayOfWeek=strftime(_time, "%A")
| `drop_dm_object_name("All_Traffic")` | fit DensityFunction count by "HourOfDay,DayOfWeek"
into smb_pdfmodel'
creation_date: '2019-05-08'
data_metadata:
data_models:
- Network_Traffic
data_source:
- Network Communications
providing_technologies:
- Splunk Stream
- Bro
name: Baseline of SMB Traffic - MLTK
id: df98763b-0b08-4281-8ef9-08db7ac572a9
version: 1
date: '2019-05-08'
description: This search is used to build a Machine Learning Toolkit (MLTK) model
to characterize the number of SMB connections observed each hour for every day of
week. By default, the search uses the last 30 days of data to build the model. The
model created by this search is then used in the corresponding detection search
to identify outliers in the number of SMB connections for that hour and day of the
week.
eli5: Create a machine-learning (ML) model to characterize the number of SMB connections
observed in your environment. This may help identify spikes in SMB traffic that
may be indicative of attackers scanning or attempting to propagate to other systems
in your environment. By default, this model is built over 30 days of data and profiles
the number of SMB connections in your environment by the hour of day/day of week
that the connections occur.
how_to_implement: You must be ingesting network traffic and populating the Network_Traffic
data model. In addition, you must have the Machine Learning Toolkit (MLTK) version
>= 4.2 installed, along with any required dependencies. To improve your results,
@@ -43,18 +21,20 @@ how_to_implement: You must be ingesting network traffic and populating the Netwo
which may give you better results. You may also want to periodically re-run this
search to rebuild the model with the latest data. More information on the algorithm
used in the search can be found at `https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms#DensityFunction`.
id: df98763b-0b08-4281-8ef9-08db7ac572a9
known_false_positives: ''
maintainers:
- company: Splunk
email: rvaldez@splunk.com
name: Rico Valdez
modification_date: '2019-05-08'
name: Baseline of SMB Traffic - MLTK
original_authors:
- company: Splunk
email: rvaldez@splunk.com
name: Rico Valdez
spec_version: 2
type: splunk
version: '1.0'
author: Rico Valdez, Splunk
search: '| tstats `security_content_summariesonly` count from datamodel=Network_Traffic
where All_Traffic.dest_port=139 OR All_Traffic.dest_port=445 OR All_Traffic.app=smb
by _time span=10m, All_Traffic.src | eval HourOfDay=strftime(_time, "%H") | eval
DayOfWeek=strftime(_time, "%A") | `drop_dm_object_name("All_Traffic")` | fit DensityFunction
count by "HourOfDay,DayOfWeek" into smb_pdfmodel'
tags:
analytics_story:
- DHS Report TA18-074A
- Disabling Security Tools
- 'Emotet Malware DHS Report TA18-201A '
- Hidden Cobra Malware
- Netsh Abuse
- Ransomware
detections:
- Processes launching netsh
- SMB Traffic Spike - MLTK
baselines/baseline_s3_deletion_by_arn.yml
-48
View File
@@ -1,48 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=DeleteBucket | spath output=arn path=userIdentity.arn
| bucket _time span=1h | stats count as apiCalls by _time, arn | stats count(apiCalls)
as numDataPoints, latest(apiCalls) as latestCount, avg(apiCalls) as avgApiCalls,
stdev(apiCalls) as stdevApiCalls by arn | table arn, latestCount, numDataPoints,
avgApiCalls, stdevApiCalls | outputlookup s3_deletion_baseline | stats count
creation_date: '2018-07-17'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search establishes, on a per-hour basis, the average and standard
deviation for the number of API calls related to deleting an S3 bucket by each user.
Also recorded is the number of data points for each user. This table is then outputted
to a lookup file to allow the detection search to operate quickly.
eli5: Use this search to create a baseline for API calls related to deleting an S3
bucket, grouped by the users who initiated this activity. It returns all logged
API calls for S3 bucket-deletion activity and then pulls out the ARN that initiated
each call. Next, it calculates the number of API calls made per ARN per hour. For
each ARN, it calculates the average and standard deviation of this count on a per-hour
basis. It also includes the number of data points for each ARN. This table is stored
in a lookup file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
id: fc0edd96-ff2b-48b0-9f1f-63eq3783fd63
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2018-07-17'
name: Baseline of S3 Bucket deletion activity by ARN
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '1.0'
baselines/baseline_security_group_activity_by_arn.yml
-49
View File
@@ -1,49 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail `security_group_api_calls` | spath output=arn path=userIdentity.arn
| bucket _time span=1h | stats count as apiCalls by _time, arn | stats count(apiCalls)
as numDataPoints, latest(apiCalls) as latestCount, avg(apiCalls) as avgApiCalls,
stdev(apiCalls) as stdevApiCalls by arn | table arn, latestCount, numDataPoints,
avgApiCalls, stdevApiCalls | outputlookup security_group_activity_baseline |
stats count
creation_date: '2018-04-17'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search establishes, on a per-hour basis, the average and the standard
deviation for the number of API calls related to security groups made by each user.
Also recorded is the number of data points for each user. This table is then outputted
to a lookup file to allow the detection search to operate quickly.
eli5: Use this search to create a baseline for API calls related to security groups
by the users who initiated this activity. It returns all logged API calls for all
security-group-related activity, pulls out the ARN that initiated each call, and
collects the `eventNames` in one-hour groupings. Next, it calculates the number
of API calls made per ARN per hour. For each ARN, it calculates the average and
standard deviation of this count on a per-hour basis. It also includes the number
of data points for each ARN. This table is stored in a lookup file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs. To add or remove API event names for security groups, edit the macro `security_group_api_calls`.
id: fc0edd96-ff2b-48b0-9f1f-63da3783fd63
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2018-04-17'
name: Baseline of Security Group Activity by ARN
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '1.0'
baselines/brand_abuse_dnstwist_domains.yml
-43
View File
@@ -1,43 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| dnstwist domainlist=domains.csv | `remove_valid_domains` | eval domain_abuse="true"
| table domain, domain_abuse | outputlookup brandMonitoring_lookup | stats count'
creation_date: '2017-06-01'
data_metadata:
data_source:
- Splunk Enterprise Security Content Update
providing_technologies:
- Splunk Enterprise
description: This search creates permutations of your existing domains, removes the
valid domain names and stores them in a specified lookup file so they can be checked
for in the associated detection searches.
eli5: This search starts with the dnstwist command consuming domains from a file called
domains.csv in the DA-ESS-SOC/lookups directory. This search then adds a domain\_abuse=true
term to each permutation, removes all the valid domain names and stores all that
information into a lookup file that is used in the associated detection search.
Alternatively domain dnstwist permutations can be calculated from domains in the
`cim_corporate_email_domains.csv` and `cim_corporate_web_domains.csv` lookups located
in **Splunk\_SA\_CIM** using argument `populate_from_cim=true`. Also an individual
domain can be passed using argument `domain=<domain>`
how_to_implement: To successfully implement this search you need to update the file
called domains.csv in the DA-ESS-SOC/lookup directory. Or `cim_corporate_email_domains.csv`
and `cim_corporate_web_domains.csv` from **Splunk\_SA\_CIM**.
id: 19f7d2ec-6028-4d01-bcdb-bda9a034c17f
known_false_positives: ''
maintainers:
- company: Splunk
email: jhernandez@splunk.com
name: Jose Hernandez
modification_date: '2018-10-08'
name: DNSTwist Domain Names
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '2.0'
baselines/count_of_assets_by_category.yml
+19
View File
@@ -0,0 +1,19 @@
name: Count of assets by category
id: dcfd6b40-42f9-469d-a433-2e53f7489ff9
version: 1
date: '2017-09-13'
description: This search shows you every asset category you have and the assets that
belong to those categories.
how_to_implement: To successfully implement this search you must first leverage the
Assets and Identity framework in Enterprise Security to populate your assets_by_str.csv
file which should then be mapped to the Identity_Management data model. The Identity_Management
data model will contain a list of known authorized company assets. Ensure that all
inventoried systems are constantly vetted and updated.
author: Bhavin Patel, Splunk
search: '| from datamodel Identity_Management.All_Assets | stats count values(nt_host)
by category | sort -count'
tags:
analytics_story:
- Asset Tracking
detections:
- Detect Unauthorized Assets by MAC address
baselines/count_of_unique_ips_connecting_to_ports.yml
+13
View File
@@ -0,0 +1,13 @@
name: Count of Unique IPs Connecting to Ports
id: 9f3bae5a-9fe3-49df-8c84-5edc51d84b7f
version: 1
date: '2017-09-13'
description: The search counts the number of times a connection was observed to each
destination port, and the number of unique source IPs connecting to them.
how_to_implement: To successfully implement this search, you must be ingesting network
traffic, and populating the Network_Traffic data model.
author: David Dorsey, Splunk
search: '| tstats `security_content_summariesonly` count dc(All_Traffic.src) as numberOfUniqueHosts
from datamodel=Network_Traffic by All_Traffic.dest_port | `drop_dm_object_name("All_Traffic")`
| sort - count'
tags: {}
baselines/create_a_list_of_approved_aws_service_accounts.yml
+21
View File
@@ -0,0 +1,21 @@
name: Create a list of approved AWS service accounts
id: fc0edc95-ff2b-48b1-5f6f-63ga3789fd43
version: 2
date: '2018-12-03'
description: This search looks for successful API activity in CloudTrail within the
last 30 days, filters out known users from the identity table, and outputs values
of users into `aws_service_accounts.csv` lookup file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Please validate the service account entires in `aws_service_accounts.csv`,
which is a lookup file created as a result of running this support search. Please
remove the entries of service accounts that are not legitimate.
author: Bhavin Patel, Splunk
search: '`cloudtrail` errorCode=success | rename userName as identity | search NOT
[inputlookup identity_lookup_expanded | fields identity] | stats count by identity
| table identity | outputlookup aws_service_accounts | stats count'
tags:
analytics_story:
- AWS User Monitoring
detections:
- Detect AWS API Activities From Unapproved Accounts
baselines/discover_dns_records.yml
+18 -44
View File
@@ -1,52 +1,26 @@
baseline:
splunk:
schedule:
cron_schedule: 0 0 1 * *
earliest_time: -30d@d
latest_time: -10m@m
search: '| inputlookup cim_corporate_email_domains.csv | inputlookup append=T
cim_corporate_web_domains.csv | inputlookup append=T cim_cloud_domains.csv |
eval domain = trim(replace(domain, "\*", "")) | join domain [|tstats `security_content_summariesonly`
count values(DNS.record_type) as type, values(DNS.answer) as answer from datamodel=Network_Resolution
where DNS.message_type=RESPONSE DNS.answer!="unknown" DNS.answer!="" by DNS.query
| rename DNS.query as query | where query!="unknown" | rex field=query "(?<domain>\w+\.\w+?)(?:$|/)"]
| makemv delim=" " answer | makemv delim=" " type | sort -count | table count,domain,type,query,answer
| outputlookup createinapp=true discovered_dns_records.csv'
creation_date: '2019-02-14'
data_metadata:
data_models:
- Network_Resolution
data_source:
- Network Communication
providing_technologies:
- Splunk Stream
- Bro
name: Discover DNS records
id: c096f721-8842-42ce-bfc7-74bd8c72b7c3
version: 1
date: '2019-02-14'
description: The search takes corporate and common cloud provider domains configured
under `cim_corporate_email_domains.csv`, `cim_corporate_web_domains.csv`, and `cloud_domains.csv`
finds their responses across the last 30 days from data in the `Network_Resolution
` datamodel, then stores the output under the `discovered_dns_records.csv` lookup
eli5: Discover the DNS records and their answers for domains owned by the company
using network traffic events. The discovered events are exported as a lookup named
`discovered_dns_records.csv`
entities:
- query
- answer
how_to_implement: To successfully implement this search, you must be ingesting DNS
logs, and populating the Network_Resolution data model. Also make sure that the
cim_corporate_web_domains and cim_corporate_email_domains lookups are populated
with the domains owned by your corporation
id: c096f721-8842-42ce-bfc7-74bd8c72b7c3
known_false_positives: 'Please vet the lookup created by this baseline search '
maintainers:
- company: Splunk
email: jhernandez@splunk.com
name: Jose Hernandez
modification_date: '2019-02-14'
name: Discover DNS records
original_authors:
- company: Splunk
email: jhernandez@splunk.com
name: Jose Hernandez
spec_version: 2
type: splunk
version: '1.0'
author: Jose Hernandez, Splunk
search: '| inputlookup cim_corporate_email_domains.csv | inputlookup append=T cim_corporate_web_domains.csv
| inputlookup append=T cim_cloud_domains.csv | eval domain = trim(replace(domain,
"\*", "")) | join domain [|tstats `security_content_summariesonly` count values(DNS.record_type)
as type, values(DNS.answer) as answer from datamodel=Network_Resolution where DNS.message_type=RESPONSE
DNS.answer!="unknown" DNS.answer!="" by DNS.query | rename DNS.query as query |
where query!="unknown" | rex field=query "(?<domain>\w+\.\w+?)(?:$|/)"] | makemv
delim=" " answer | makemv delim=" " type | sort -count | table count,domain,type,query,answer
| outputlookup createinapp=true discovered_dns_records.csv'
tags:
analytics_story:
- DNS Hijacking
detections:
- DNS record changed
baselines/dnstwist_domain_names.yml
+21
View File
@@ -0,0 +1,21 @@
name: DNSTwist Domain Names
id: 19f7d2ec-6028-4d01-bcdb-bda9a034c17f
version: 2
date: '2018-10-08'
description: This search creates permutations of your existing domains, removes the
valid domain names and stores them in a specified lookup file so they can be checked
for in the associated detection searches.
how_to_implement: To successfully implement this search you need to update the file
called domains.csv in the DA-ESS-SOC/lookup directory. Or `cim_corporate_email_domains.csv`
and `cim_corporate_web_domains.csv` from **Splunk\_SA\_CIM**.
author: David Dorsey, Splunk
search: '| dnstwist domainlist=domains.csv | `remove_valid_domains` | eval domain_abuse="true"
| table domain, domain_abuse | outputlookup brandMonitoring_lookup | stats count'
tags:
analytics_story:
- Brand Monitoring
- Suspicious Emails
detections:
- Monitor Email For Brand Abuse
- Monitor DNS For Brand Abuse
- Monitor Web Traffic For Brand Abuse
baselines/identify_ports_on_network.yml
-39
View File
@@ -1,39 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` count dc(All_Traffic.src) as numberOfUniqueHosts
from datamodel=Network_Traffic by All_Traffic.dest_port | `drop_dm_object_name("All_Traffic")`
| sort - count'
creation_date: '2017-06-24'
data_metadata:
data_models:
- Network_Traffic
data_source:
- Network Communication
providing_technologies:
- Splunk Stream
- Bro
description: The search counts the number of times a connection was observed to each
destination port, and the number of unique source IPs connecting to them.
eli5: For each port being accessed on the network, this search gives the total number
of connections observed, and the number of unique IP addresses making those connections.
how_to_implement: To successfully implement this search, you must be ingesting network
traffic, and populating the Network_Traffic data model.
id: 9f3bae5a-9fe3-49df-8c84-5edc51d84b7f
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2017-09-13'
name: Count of Unique IPs Connecting to Ports
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/identify_systems_creating_rdp_traffic.yml
-38
View File
@@ -1,38 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` count from datamodel=Network_Traffic where All_Traffic.dest_port=3389
by All_Traffic.src | `drop_dm_object_name("All_Traffic")` | sort - count'
creation_date: '2017-04-24'
data_metadata:
data_models:
- Network_Traffic
data_source:
- Network Communication
providing_technologies:
- Splunk Stream
- Bro
description: This search counts the numbers of times the system has generated remote
desktop traffic.
eli5: This search counts the numbers of times the system has tried to connect to another
system on TCP/3389, the default port used for RDP traffic.
how_to_implement: To successfully implement this search, you must ingest network traffic
and populate the Network_Traffic data model.
id: 5cdda34f-4caf-4128-a713-0837fc48b67a
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2017-09-15'
name: Identify Systems Creating Remote Desktop Traffic
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/identify_systems_creating_remote_desktop_traffic.yml
+13
View File
@@ -0,0 +1,13 @@
name: Identify Systems Creating Remote Desktop Traffic
id: 5cdda34f-4caf-4128-a713-0837fc48b67a
version: 1
date: '2017-09-15'
description: This search counts the numbers of times the system has generated remote
desktop traffic.
how_to_implement: To successfully implement this search, you must ingest network traffic
and populate the Network_Traffic data model.
author: David Dorsey, Splunk
search: '| tstats `security_content_summariesonly` count from datamodel=Network_Traffic
where All_Traffic.dest_port=3389 by All_Traffic.src | `drop_dm_object_name("All_Traffic")`
| sort - count'
tags: {}
baselines/identify_systems_receiving_rdp_traffic.yml
-39
View File
@@ -1,39 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` count from datamodel=Network_Traffic where All_Traffic.dest_port=3389
by All_Traffic.dest | `drop_dm_object_name("All_Traffic")` | sort - count'
creation_date: '2017-04-24'
data_metadata:
data_models:
- Network_Traffic
data_source:
- Network Communication
providing_technologies:
- Splunk Stream
- Bro
description: This search counts the numbers of times the system has created remote
desktop traffic
eli5: This search counts the numbers of times the system has received a connection
to TCP/ 3389, the default port used for RDP traffic.
how_to_implement: To successfully implement this search you must ingest network traffic
and populate the Network_Traffic data model. If a system receives a lot of remote
desktop traffic, you can apply the category common_rdp_destination to it.
id: baaeea15-fe8a-4090-92c2-5b60943bb608
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2017-09-15'
name: Identify Systems Receiving Remote Desktop Traffic
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/identify_systems_receiving_remote_desktop_traffic.yml
+14
View File
@@ -0,0 +1,14 @@
name: Identify Systems Receiving Remote Desktop Traffic
id: baaeea15-fe8a-4090-92c2-5b60943bb608
version: 1
date: '2017-09-15'
description: This search counts the numbers of times the system has created remote
desktop traffic
how_to_implement: To successfully implement this search you must ingest network traffic
and populate the Network_Traffic data model. If a system receives a lot of remote
desktop traffic, you can apply the category common_rdp_destination to it.
author: David Dorsey, Splunk
search: '| tstats `security_content_summariesonly` count from datamodel=Network_Traffic
where All_Traffic.dest_port=3389 by All_Traffic.dest | `drop_dm_object_name("All_Traffic")`
| sort - count'
tags: {}
baselines/identify_systems_using_remote_desktop.yml
+9 -39
View File
@@ -1,43 +1,13 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` count from datamodel=Endpoint.Processes where
Processes.process_name="*mstsc.exe*" by Processes.dest Processes.process_name
| `drop_dm_object_name(Processes)` | sort - count'
creation_date: '2017-04-18'
data_metadata:
data_models:
- Endpoint
data_source:
- Endpoint Intel
providing_technologies:
- Carbon Black Response
- CrowdStrike Falcon
- Sysmon
- Tanium
- Ziften
name: Identify Systems Using Remote Desktop
id: 063dfe9f-b1d7-4254-a16d-1e2e7eadd6a8
version: 1
date: '2019-04-01'
description: This search counts the numbers of times the remote desktop process, mstsc.exe,
has run on each system.
eli5: This search counts the numbers of times the remote desktop process, mstsc.exe,
has run on each system. It does this by looking for the process name in the Endpoint
data model.
how_to_implement: To successfully implement this search you must be ingesting endpoint
data that records process activity.
id: 063dfe9f-b1d7-4254-a16d-1e2e7eadd6a8
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2019-04-01'
name: Identify Systems Using Remote Desktop
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '| tstats `security_content_summariesonly` count from datamodel=Endpoint.Processes
where Processes.process_name="*mstsc.exe*" by Processes.dest Processes.process_name
| `drop_dm_object_name(Processes)` | sort - count'
tags: {}
baselines/monitor_successful_backups.yml
+13 -34
View File
@@ -1,39 +1,18 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype="netbackup_logs" "Disk/Partition backup completed successfully."
| bucket _time span=1d | stats dc(COMPUTERNAME) as count values(COMPUTERNAME)
as dest by _time, MESSAGE
creation_date: '2017-08-24'
data_metadata:
data_source:
- Backup Logs
data_sourcetypes:
- netbackup_logs
providing_technologies:
- Netbackup
name: Monitor Successful Backups
id: b4d0dfb2-2195-4f6e-93a3-48468ed9734e
version: 1
date: '2017-09-12'
description: This search is intended to give you a feel for how often successful backups
are conducted in your environment. Fluctuations in these numbers will allow you
to determine when you should investigate.
eli5: This search gives you the count and the hostname of all the systems that had
a successful backup each day.
how_to_implement: To successfully implement this search you must be ingesting your
backup logs.
id: b4d0dfb2-2195-4f6e-93a3-48468ed9734e
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2017-09-12'
name: Monitor Successful Backups
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '`netbackup` "Disk/Partition backup completed successfully." | bucket _time
span=1d | stats dc(COMPUTERNAME) as count values(COMPUTERNAME) as dest by _time,
MESSAGE'
tags:
analytics_story:
- Monitor Backup Solution
detections:
- Unsuccessful Netbackup backups
baselines/monitor_successful_windows_updates.yml
-38
View File
@@ -1,38 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` dc(Updates.dest) as count FROM datamodel=Updates
where Updates.vendor_product="Microsoft Windows" AND Updates.status=installed
by _time span=1d'
creation_date: '2017-08-24'
data_metadata:
data_models:
- Updates
data_source:
- Windows Updates
providing_technologies:
- Microsoft Windows
description: This search is intended to give you a feel for how often successful Windows
updates are applied in your environments. Fluctuations in these numbers will allow
you to determine when you should be concerned.
eli5: This search gives you the count and name of all the systems that had a successful
update applied each day
how_to_implement: You must be ingesting your Windows Update Logs
id: 6a80535c-86a6-4b54-894c-4b446d0c701d
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2017-09-14'
name: Windows Updates Install Successes
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/monitor_unsuccessful_backups.yml
+12 -34
View File
@@ -1,39 +1,17 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype="netbackup_logs" "An error occurred, failed to backup." | bucket
_time span=1d | stats dc(COMPUTERNAME) as count values(COMPUTERNAME) as dest
by _time, MESSAGE
creation_date: '2017-08-24'
data_metadata:
data_source:
- Backup Logs
data_sourcetypes:
- netbackup_logs
providing_technologies:
- Netbackup
name: Monitor Unsuccessful Backups
id: b2178fed-592f-492b-b851-74161678aa56
version: 1
date: '2017-09-12'
description: This search is intended to give you a feel for how often backup failures
happen in your environments. Fluctuations in these numbers will allow you to determine
when you should investigate.
eli5: This search gives you the count and hostname of all the systems that had a backup
failure each day
how_to_implement: To successfully implement this search you must be ingesting your
backup logs.
id: b2178fed-592f-492b-b851-74161678aa56
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2017-09-12'
name: Monitor Unsuccessful Backups
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '`netbackup` "An error occurred, failed to backup." | bucket _time span=1d
| stats dc(COMPUTERNAME) as count values(COMPUTERNAME) as dest by _time, MESSAGE'
tags:
analytics_story:
- Monitor Backup Solution
detections:
- Unsuccessful Netbackup backups
baselines/monitor_unsuccessful_windows_updates.yml
-38
View File
@@ -1,38 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` dc(Updates.dest) as count FROM datamodel=Updates
where Updates.vendor_product="Microsoft Windows" AND Updates.status=failure
by _time span=1d'
creation_date: '2017-08-24'
data_metadata:
data_models:
- Updates
data_source:
- Windows Updates
providing_technologies:
- Microsoft Windows
description: This search is intended to give you a feel for how often Windows updates
fail to install in your environment. Fluctuations in these numbers will allow you
to determine when you should be concerned.
eli5: This search gives you the count of the number of systems that attempted and
failed to install a Windows update each day.
how_to_implement: You must be ingesting your Windows Update Logs
id: 6a4dbd1b-4502-4a11-943a-82b5ae7a42d7
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2017-09-14'
name: Windows Updates Install Failures
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/previously_seen_api_call_per_user_roles_in_cloudtrail.yml
+22
View File
@@ -0,0 +1,22 @@
name: Previously seen API call per user roles in CloudTrail
id: fc0edc95-fq2c-48b0-9f6f-63da3289fd03
version: 1
date: '2018-04-16'
description: This search looks for successful API calls made by different user roles,
then creates a baseline of the earliest and latest times we have encountered this
user role. It also returns the name of the API call in our dataset--grouped by user
role and name of the API call--that occurred within the last 30 days. In this support
search, we are only looking for events where the user identity is Assumed Role.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Please validate the user role entries in `previously_seen_api_calls_from_user_roles.csv`,
which is a lookup file created as a result of running this support search.
author: Bhavin Patel, Splunk
search: '`cloudtrail` eventType=AwsApiCall errorCode=success userIdentity.type=AssumedRole
| stats earliest(_time) as earliest latest(_time) as latest by userName eventName
| outputlookup previously_seen_api_calls_from_user_roles | stats count'
tags:
analytics_story:
- AWS User Monitoring
detections:
- Detect new API calls from user roles
baselines/previously_seen_aws_cross_account_activity.yml
+15 -38
View File
@@ -1,45 +1,22 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=AssumeRole | spath output=requestingAccountId
path=userIdentity.accountId | spath output=requestedAccountId path=resources{}.accountId
| search requestingAccountId=* | where requestingAccountId!=requestedAccountId
| stats earliest(_time) as firstTime latest(_time) as lastTime by requestingAccountId,
requestedAccountId | outputlookup previously_seen_aws_cross_account_activity
| stats count
creation_date: '2018-06-04'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
name: Previously Seen AWS Cross Account Activity
id: 1cc22b09-c867-416e-a511-cb36ac44aee2
version: 1
date: '2018-06-04'
description: This search looks for **AssumeRole** events where the requesting account
differs from the requested account, then writes these relationships to a lookup
file.
eli5: In this support search, we look for **AssumeRole** events where the requesting
account is different from the requested account. The first and last times these
events are seen are written to a lookup file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Validate the user name entries in `previously_seen_aws_cross_account_activity.csv`,
a lookup file created by this support search.
id: 1cc22b09-c867-416e-a511-cb36ac44aee2
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-06-04'
name: Previously Seen AWS Cross Account Activity
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '`cloudtrail` eventName=AssumeRole | spath output=requestingAccountId path=userIdentity.accountId
| spath output=requestedAccountId path=resources{}.accountId | search requestingAccountId=*
| where requestingAccountId!=requestedAccountId | stats earliest(_time) as firstTime
latest(_time) as lastTime by requestingAccountId, requestedAccountId | outputlookup
previously_seen_aws_cross_account_activity | stats count'
tags:
analytics_story:
- AWS Cross Account Activity
detections:
- AWS Cross Account Activity From Previously Unseen Account
baselines/previously_seen_aws_provisioning_activity_sources.yml
+23
View File
@@ -0,0 +1,23 @@
name: Previously Seen AWS Provisioning Activity Sources
id: ac88e6a0-4fba-4dfd-b7b9-8964df7d1aee
version: 1
date: '2018-03-16'
description: This search builds a table of the first and last times seen for every
IP address (along with its physical location) previously associated with cloud-provisioning
activity. This is broadly defined as any event that runs or creates something.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs.
author: David Dorsey, Splunk
search: '`cloudtrail` (eventName=Run* OR eventName=Create*) | iplocation sourceIPAddress
| stats earliest(_time) as firstTime, latest(_time) as lastTime by sourceIPAddress,
City, Region, Country | outputlookup previously_seen_provisioning_activity_src.csv
| stats count'
tags:
analytics_story:
- AWS Suspicious Provisioning Activities
detections:
- AWS Cloud Provisioning From Previously Unseen IP Address
- AWS Cloud Provisioning From Previously Unseen City
- AWS Cloud Provisioning From Previously Unseen Country
- AWS Cloud Provisioning From Previously Unseen Region
baselines/previously_seen_aws_regions.yml
+20
View File
@@ -0,0 +1,20 @@
name: Previously Seen AWS Regions
id: fc0edc95-ff2b-48b0-9f6f-63da3789fd63
version: 1
date: '2018-01-08'
description: This search looks for CloudTrail events where an AWS instance is started
and creates a baseline of most recent time (latest) and the first time (earliest)
we've seen this region in our dataset grouped by the value awsRegion for the last
30 days
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
author: Bhavin Patel, Splunk
search: '`cloudtrail` StartInstances | stats earliest(_time) as earliest latest(_time)
as latest by awsRegion | outputlookup previously_seen_aws_regions.csv | stats count'
tags:
analytics_story:
- AWS Cryptomining
- Suspicious AWS EC2 Activities
detections:
- EC2 Instance Started In Previously Unseen Region
baselines/previously_seen_aws_users.yml
-56
View File
@@ -1,56 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: 0 0 1 * *
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=ConsoleLogin | rename userIdentity.arn
as user | iplocation src | eval City=if(City LIKE "",src,City),Region=if(Region
LIKE "",src,Region) | stats earliest(_time) as firstTime latest(_time) as lastTime
by user src City Region Country | outputlookup previously_seen_users_console_logins.csv
| stats count
creation_date: '2018-02-23'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search looks for CloudTrail events where a user logs into the console,
then creates a baseline of the latest and earliest times, City, Region, and Country
we have encountered this user in our dataset, grouped by ARN, within the last 30
days.
eli5: In this support search, we look for console login events by a particular user
and create an initial baseline cache for the previous 30 days, including the earliest
and latest times, City, Region, and Country a particular user ARN is seen in our
dataset, grouped by the ARN value. In cases where City and Region cannot be determined,
the source IP address is substituted for these values.
entities:
- user
- src
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Please validate the user name entries in `previously_seen_users_console_logins.csv`,
which is a lookup file created as a result of running this support search.
id: fc0edc95-ff2b-48b0-9f6f-63da3789fd03
known_false_positives: n/a
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
- company: Splunk
email: jbrewer@splunk.com
name: Jason Brewer
modification_date: '2018-04-30'
name: Previously seen users in CloudTrail
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
- company: Splunk
email: jbrewer@splunk.com
name: Jason Brewer
spec_version: 2
type: splunk
version: '1.1'
baselines/previously_seen_cloud_compute_creations_by_user.yml
+14 -41
View File
@@ -1,45 +1,18 @@
baseline:
splunk:
lookups:
- previously_seen_cloud_compute_creations_by_user
macros:
- previously_seen_cloud_compute_creations_by_user_input_filter
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: '| tstats earliest(_time) as firstTime, latest(_time) as lastTime from
datamodel=Cloud_Infrastructure.Compute where Compute.action=run `previously_seen_cloud_compute_creations_by_user_input_filter`
by Compute.src_user | `drop_dm_object_name("Compute")` | outputlookup previously_seen_cloud_compute_creations_by_user
| stats count'
creation_date: '2019-10-03'
data_metadata:
data_models:
- Cloud_Infrastructure
data_source:
- Cloud Infrastructure Logs
providing_technologies:
- AWS
- Azure
- GCP
name: Previously Seen Cloud Compute Creations By User
id: 9fa1c205-4e08-4681-bb1b-d0943e734b85
version: 1
date: '2018-03-15'
description: This search builds a table of previously seen users that have launched
a cloud compute instance.
eli5: In this support search, we create a table of the earliest and latest time for
each user that has created a cloud compute instance.
how_to_implement: You must be ingesting the approrpiate cloud infrastructure logs
and have the Security Research cloud data model installed.
id: 9fa1c205-4e08-4681-bb1b-d0943e734b85
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-03-15'
name: Previously Seen Cloud Compute Creations By User
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '| tstats earliest(_time) as firstTime, latest(_time) as lastTime from datamodel=Cloud_Infrastructure.Compute
where Compute.action=run `previously_seen_cloud_compute_creations_by_user_input_filter`
by Compute.src_user | `drop_dm_object_name("Compute")` | outputlookup previously_seen_cloud_compute_creations_by_user
| stats count'
tags:
analytics_story:
- Cloud Cryptomining
detections:
- Cloud Compute Instance Created By Previously Unseen User
baselines/previously_seen_cloud_compute_images.yml
+14 -41
View File
@@ -1,45 +1,18 @@
baseline:
splunk:
lookups:
- previously_seen_cloud_compute_images
macros:
- previously_seen_cloud_compute_image_input_filter
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: '| tstats earliest(_time) as firstTime, latest(_time) as lastTime from
datamodel=Cloud_Infrastructure.Compute where Compute.action=run `previously_seen_cloud_compute_image_input_filter`
by Compute.image_id | `drop_dm_object_name("Compute")` | outputlookup previously_seen_cloud_compute_images
| stats count'
creation_date: '2019-10-03'
data_metadata:
data_models:
- Cloud_Infrastructure
data_source:
- Cloud Infrastructure Logs
providing_technologies:
- AWS
- Azure
- GCP
name: Previously Seen Cloud Compute Images
id: 3782ad10-5ce2-46e2-b9c4-1de9ecd3aecc
version: 1
date: '2018-03-12'
description: This search builds a table of previously seen images used to launch cloud
compute instances
eli5: In this support search, we create a table of the earliest and latest time for
each image id that has been seen. This table is then outputted to a csv file.
how_to_implement: You must be ingesting the approrpiate cloud infrastructure logs
and have the Security Research cloud data model installed.
id: 3782ad10-5ce2-46e2-b9c4-1de9ecd3aecc
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-03-12'
name: Previously Seen Cloud Compute Images
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '| tstats earliest(_time) as firstTime, latest(_time) as lastTime from datamodel=Cloud_Infrastructure.Compute
where Compute.action=run `previously_seen_cloud_compute_image_input_filter` by Compute.image_id
| `drop_dm_object_name("Compute")` | outputlookup previously_seen_cloud_compute_images
| stats count'
tags:
analytics_story:
- Cloud Cryptomining
detections:
- Cloud Compute Instance Created With Previously Unseen Image
baselines/previously_seen_cloud_compute_instance_types.yml
+14 -44
View File
@@ -1,48 +1,18 @@
baseline:
splunk:
lookups:
- previously_seen_cloud_compute_instance_types
macros:
- previously_seen_cloud_compute_instance_types_input_filter
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: '| tstats earliest(_time) as firstTime, latest(_time) as lastTime from
datamodel=Cloud_Infrastructure.Compute where Compute.action=run `previously_seen_cloud_compute_instance_types_input_filter`
by Compute.instance_type | `drop_dm_object_name("Compute")` | outputlookup previously_seen_cloud_compute_instance_types
| stats count'
creation_date: '2019-10-03'
data_metadata:
data_models:
- Cloud_Infrastructure
data_source:
- Cloud Infrastructure Logs
providing_technologies:
- AWS
- Azure
- GCP
name: Previously Seen Cloud Compute Instance Types
id: 0ef13d46-164e-4cf5-816e-b3c0df170d00
version: 1
date: '2019-10-03'
description: This search builds a table of previously seen cloud compute instance
types
eli5: In this support search, we create a table of the first time `firstTime` and
most recent time `lastTime` that the compute type has been seen in our dataset.
We only look for those events where an instance has been created. All of these entries
will be added to the `previously_seen_cloud_compute_instance_types` lookup file,
which will act as a baseline for detections.
how_to_implement: You must be ingesting the approrpiate cloud infrastructure logs
and have the Security Research cloud data model installed.
id: 0ef13d46-164e-4cf5-816e-b3c0df170d00
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2019-10-03'
name: Previously Seen Cloud Compute Instance Types
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '| tstats earliest(_time) as firstTime, latest(_time) as lastTime from datamodel=Cloud_Infrastructure.Compute
where Compute.action=run `previously_seen_cloud_compute_instance_types_input_filter`
by Compute.instance_type | `drop_dm_object_name("Compute")` | outputlookup previously_seen_cloud_compute_instance_types
| stats count'
tags:
analytics_story:
- Cloud Cryptomining
detections:
- Cloud Compute Instance Created With Previously Unseen Instance Type
baselines/previously_seen_cloud_regions.yml
+20
View File
@@ -0,0 +1,20 @@
name: Previously Seen Cloud Regions
id: b5e232db-dec6-4db8-aaa1-dd5474521e40
version: 1
date: '2019-10-02'
description: This search looks for cloud compute events where a compute instance is
started and creates a baseline of most recent time, `lastTime` and the first time
`firstTime` we've seen this region in our dataset grouped by the region for the
last 30 days
how_to_implement: You must be ingesting the approrpiate cloud infrastructure logs
and have the Security Research cloud data model installed.
author: David Dorsey, Splunk
search: '| tstats earliest(_time) as firstTime, latest(_time) as lastTime from datamodel=Cloud_Infrastructure.Compute
where Compute.action=start `previously_seen_cloud_regions_input_filter` by Compute.region
| `drop_dm_object_name("Compute")` | outputlookup previously_seen_cloud_regions
| stats count'
tags:
analytics_story:
- Cloud Cryptomining
detections:
- Cloud Compute Instance Started In Previously Unused Region
baselines/previously_seen_cmd_arguments.yml
-48
View File
@@ -1,48 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` min(_time) as firstTime max(_time) as lastTime
from datamodel=Endpoint.Processes where Processes.process_name=cmd.exe AND Processes.process="*
/c *" by Processes.process | `drop_dm_object_name(Processes)`'
creation_date: '2018-04-09'
data_metadata:
data_models:
- Endpoint
data_source:
- Endpoint Intel
providing_technologies:
- Carbon Black Response
- CrowdStrike Falcon
- Sysmon
- Tanium
- Ziften
description: This search looks for command-line arguments where `cmd.exe /c` is used
to execute a program, then creates a baseline of the earliest and latest times we
have encountered this command-line argument in our dataset within the last 30 days.
eli5: In this support search, we look for command-line arguments using the parameter
`/c` to execute processes and create an initial baseline cache for the previous
30 days. This will include the earliest and latest times a particular command-line
argument is seen in our dataset, grouped by the command-line value.
how_to_implement: You must be ingesting data that records process activity from your
hosts to populate the Endpoint data model in the Processes node. You must be ingesting
logs with both the process name and command line from your endpoints. The complete
process name with command-line arguments are mapped to the "process" field in the
Endpoint data model.
id: fc0edc95-ff2b-48b0-9f6f-63da3789fd23
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2019-03-01'
name: Previously seen command line arguments
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '2.0'
baselines/previously_seen_command_line_arguments.yml
+30
View File
@@ -0,0 +1,30 @@
name: Previously seen command line arguments
id: fc0edc95-ff2b-48b0-9f6f-63da3789fd23
version: 2
date: '2019-03-01'
description: This search looks for command-line arguments where `cmd.exe /c` is used
to execute a program, then creates a baseline of the earliest and latest times we
have encountered this command-line argument in our dataset within the last 30 days.
how_to_implement: You must be ingesting data that records process activity from your
hosts to populate the Endpoint data model in the Processes node. You must be ingesting
logs with both the process name and command line from your endpoints. The complete
process name with command-line arguments are mapped to the "process" field in the
Endpoint data model.
author: Bhavin Patel, Splunk
search: '| tstats `security_content_summariesonly` min(_time) as firstTime max(_time)
as lastTime from datamodel=Endpoint.Processes where Processes.process_name=cmd.exe
AND Processes.process="* /c *" by Processes.process | `drop_dm_object_name(Processes)`'
tags:
analytics_story:
- DHS Report TA18-074A
- Disabling Security Tools
- Hidden Cobra Malware
- Netsh Abuse
- Orangeworm Attack Group
- Possible Backdoor Activity Associated With MUDCARP Espionage Campaigns
- Suspicious Command-Line Executions
- Suspicious MSHTA Activity
detections:
- Detect Prohibited Applications Spawning cmd.exe
- Processes launching netsh
- First time seen command line argument
baselines/previously_seen_ec2_amis.yml
+13 -35
View File
@@ -1,40 +1,18 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=RunInstances errorCode=success | rename
requestParameters.instancesSet.items{}.imageId as amiID | stats earliest(_time)
as firstTime latest(_time) as lastTime by amiID | outputlookup previously_seen_ec2_amis.csv
| stats count
creation_date: '2018-03-12'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
name: Previously Seen EC2 AMIs
id: bb1bd99d-1e93-45f1-9571-cfed42d372b9
version: 1
date: '2018-03-12'
description: This search builds a table of previously seen AMIs used to launch EC2
instances
eli5: In this support search, we create a table of the earliest and latest time that
a specific AMI ID has been seen. This table is then outputted to a csv file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
id: bb1bd99d-1e93-45f1-9571-cfed42d372b9
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-03-12'
name: Previously Seen EC2 AMIs
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '`cloudtrail` eventName=RunInstances errorCode=success | rename requestParameters.instancesSet.items{}.imageId
as amiID | stats earliest(_time) as firstTime latest(_time) as lastTime by amiID
| outputlookup previously_seen_ec2_amis.csv | stats count'
tags:
analytics_story:
- AWS Cryptomining
detections:
- EC2 Instance Started With Previously Unseen AMI
baselines/previously_seen_ec2_instance_types.yml
+14 -37
View File
@@ -1,41 +1,18 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=RunInstances errorCode=success | rename
requestParameters.instanceType as instanceType | fillnull value="m1.small" instanceType
| stats earliest(_time) as earliest latest(_time) as latest by instanceType
| outputlookup previously_seen_ec2_instance_types.csv | stats count
creation_date: '2018-03-08'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
name: Previously Seen EC2 Instance Types
id: b8f029f2-65a6-4d76-be98-dad1c9d59c45
version: 1
date: '2018-03-08'
description: This search builds a table of previously seen EC2 instance types
eli5: In this support search, we create a table of the earliest and latest time that
a specific EC2 instance type has been seen. The instanceType request field is not
required and defaults to m1.small, so any time this field is null, the search defaults
the field to m1.small. This table is then outputted to a csv file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
id: b8f029f2-65a6-4d76-be98-dad1c9d59c45
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-03-08'
name: Previously Seen EC2 Instance Types
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '`cloudtrail` eventName=RunInstances errorCode=success | rename requestParameters.instanceType
as instanceType | fillnull value="m1.small" instanceType | stats earliest(_time)
as earliest latest(_time) as latest by instanceType | outputlookup previously_seen_ec2_instance_types.csv
| stats count'
tags:
analytics_story:
- AWS Cryptomining
detections:
- EC2 Instance Started With Previously Unseen Instance Type
baselines/previously_seen_ec2_launches_by_user.yml
+14 -35
View File
@@ -1,40 +1,19 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=RunInstances errorCode=success | rename
userIdentity.arn as arn | stats earliest(_time) as firstTime latest(_time) as
lastTime by arn | outputlookup previously_seen_ec2_launches_by_user.csv | stats
count
creation_date: '2018-03-15'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
name: Previously Seen EC2 Launches By User
id: 6c767ac0-0906-4355-9a83-927f5ee7bdad
version: 1
date: '2018-03-15'
description: This search builds a table of previously seen ARNs that have launched
a EC2 instance.
eli5: In this support search, we create a table of the earliest and latest times that
an ARN has launched a EC2 instance. This table is then outputted to a csv file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs.
id: 6c767ac0-0906-4355-9a83-927f5ee7bdad
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-03-15'
name: Previously Seen EC2 Launches By User
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
author: David Dorsey, Splunk
search: '`cloudtrail` eventName=RunInstances errorCode=success | rename userIdentity.arn
as arn | stats earliest(_time) as firstTime latest(_time) as lastTime by arn | outputlookup
previously_seen_ec2_launches_by_user.csv | stats count'
tags:
analytics_story:
- AWS Cryptomining
- Suspicious AWS EC2 Activities
detections:
- EC2 Instance Started With Previously Unseen User
baselines/previously_seen_ec2_modifications.yml
-42
View File
@@ -1,42 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail `ec2_modification_api_calls` errorCode=success | spath
output=arn userIdentity.arn | stats earliest(_time) as firstTime latest(_time)
as lastTime by arn | outputlookup previously_seen_ec2_modifications_by_user
| stats count
creation_date: '2018-04-05'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search builds a table of previously seen ARNs that have launched
a EC2 instance.
eli5: In this support search, we create a table of the earliest and latest times that
an ARN has modified a EC2 instance. The list of APIs that modify an EC2 are defined
in the `ec2_modification_api_calls` macro for ease of use. This table is then outputted
to a file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs. To add or remove APIs that modify an EC2 instance, edit the macro `ec2_modification_api_calls`.
id: 4d69091b-d975-4267-85df-888bd41034eb
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-04-05'
name: Previously Seen EC2 Modifications By User
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/previously_seen_ec2_modifications_by_user.yml
+18
View File
@@ -0,0 +1,18 @@
name: Previously Seen EC2 Modifications By User
id: 4d69091b-d975-4267-85df-888bd41034eb
version: 1
date: '2018-04-05'
description: This search builds a table of previously seen ARNs that have launched
a EC2 instance.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS version (4.4.0 or later), then configure your CloudTrail
inputs. To add or remove APIs that modify an EC2 instance, edit the macro `ec2_modification_api_calls`.
author: David Dorsey, Splunk
search: '`cloudtrail` `ec2_modification_api_calls` errorCode=success | spath output=arn
userIdentity.arn | stats earliest(_time) as firstTime latest(_time) as lastTime
by arn | outputlookup previously_seen_ec2_modifications_by_user | stats count'
tags:
analytics_story:
- Unusual AWS EC2 Modifications
detections:
- EC2 Instance Modified With Previously Unseen User
baselines/previously_seen_provisioning_activity_src.yml
-43
View File
@@ -1,43 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -90d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail (eventName=Run* OR eventName=Create*) | iplocation
sourceIPAddress | stats earliest(_time) as firstTime, latest(_time) as lastTime
by sourceIPAddress, City, Region, Country | outputlookup previously_seen_provisioning_activity_src.csv
| stats count
creation_date: '2018-03-16'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search builds a table of the first and last times seen for every
IP address (along with its physical location) previously associated with cloud-provisioning
activity. This is broadly defined as any event that runs or creates something.
eli5: This search includes any event name that begins with "run" or "create," and
then determines the first and last time these events were seen for each IP address
that initiated the action. The search then consults a **GeoIP** database to determine
the physical location of this IP address. This table outputs to a file.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs.
id: ac88e6a0-4fba-4dfd-b7b9-8964df7d1aee
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-03-16'
name: Previously Seen AWS Provisioning Activity Sources
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/previously_seen_running_windows_services.yml
+20
View File
@@ -0,0 +1,20 @@
name: Previously Seen Running Windows Services
id: 64ce0ade-cb01-4678-bddd-d31c0b175394
version: 2
date: '2020-01-13'
description: This collects the services that have been started across your entire
enterprise.
how_to_implement: While this search does not require you to adhere to Splunk CIM,
you must be ingesting your Windows security-event logs for it to execute successfully.
Please ensure that the Splunk Add-on for Microsoft Windows is version 5.0.0 or above.
author: David Dorsey, Splunk
search: '`wineventlog_system` signature_id=7036 | rename param1 as service_name |
rename param2 as action | search action="running" | stats earliest(_time) as firstTime,
latest(_time) as lastTime by service_name | outputlookup previously_seen_running_windows_services
| stats count'
tags:
analytics_story:
- Orangeworm Attack Group
- Windows Service Abuse
detections:
- First Time Seen Running Windows Service
baselines/previously_seen_s3_bucket_access_by_remote_ip.yml
+21
View File
@@ -0,0 +1,21 @@
name: Previously seen S3 bucket access by remote IP
id: fc0edc15-fq2c-48b0-9f6f-63qa1281fd03
version: 1
date: '2018-06-28'
description: This search looks for successful access to S3 buckets from remote IP
addresses, then creates a baseline of the earliest and latest times we have encountered
this remote IP within the last 30 days. In this support search, we are only looking
for S3 access events where the HTTP response code from AWS is "200"
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your S3 access-logs
inputs. You must validate the remote IP and bucket name entries in `previously_seen_S3_access_from_remote_ip.csv`,
which is a lookup file created as a result of running this support search.
author: Bhavin Patel, Splunk
search: '`aws_s3_accesslogs` http_status=200 | stats earliest(_time) as earliest
latest(_time) as latest by bucket_name remote_ip | outputlookup previously_seen_S3_access_from_remote_ip
| stats count'
tags:
analytics_story:
- Suspicious AWS S3 Activities
detections:
- Detect S3 access from a new IP
baselines/previously_seen_s3_remote_ip.yml
-45
View File
@@ -1,45 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:s3:accesslogs http_status=200 | stats earliest(_time)
as earliest latest(_time) as latest by bucket_name remote_ip | outputlookup
previously_seen_S3_access_from_remote_ip | stats count
creation_date: '2018-06-28'
data_metadata:
data_source:
- AWS S3 Access logs
data_sourcetypes:
- aws:s3:accesslogs
providing_technologies:
- AWS
description: This search looks for successful access to S3 buckets from remote IP
addresses, then creates a baseline of the earliest and latest times we have encountered
this remote IP within the last 30 days. In this support search, we are only looking
for S3 access events where the HTTP response code from AWS is "200"
eli5: In this support search, we are looking for successful S3 bucket-access attempts
made from remote IPs. The intent is to create an initial baseline cache of remote
IP addresses per bucket name for the previous 30 days--including the earliest and
latest times seen in our dataset--grouped by the value of remote IP and the name
of the S3 bucket.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your S3 access-logs
inputs. You must validate the remote IP and bucket name entries in `previously_seen_S3_access_from_remote_ip.csv`,
which is a lookup file created as a result of running this support search.
id: fc0edc15-fq2c-48b0-9f6f-63qa1281fd03
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2018-06-28'
name: Previously seen S3 bucket access by remote IP
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '1.0'
baselines/previously_seen_user_roles.yml
-47
View File
@@ -1,47 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventType=AwsApiCall errorCode=success userIdentity.type=AssumedRole
| stats earliest(_time) as earliest latest(_time) as latest by userName eventName
| outputlookup previously_seen_api_calls_from_user_roles | stats count
creation_date: '2018-04-01'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search looks for successful API calls made by different user roles,
then creates a baseline of the earliest and latest times we have encountered this
user role. It also returns the name of the API call in our dataset--grouped by user
role and name of the API call--that occurred within the last 30 days. In this support
search, we are only looking for events where the user identity is Assumed Role.
eli5: In this support search, we are looking for successful API calls made by user
roles within your AWS infrastructure. The intent is to create an initial baseline
cache of names of the API calls per security role for the previous 30 days--including
the earliest and latest times seen in our dataset--grouped by the value of user
role and the name of the API call. It is also worth noting that the role of a particular
user is parsed as "userName" in the CloudTrail logs.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Please validate the user role entries in `previously_seen_api_calls_from_user_roles.csv`,
which is a lookup file created as a result of running this support search.
id: fc0edc95-fq2c-48b0-9f6f-63da3289fd03
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2018-04-16'
name: Previously seen API call per user roles in CloudTrail
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
spec_version: 2
type: splunk
version: '1.0'
baselines/previously_seen_users_in_cloudtrail.yml
+25
View File
@@ -0,0 +1,25 @@
name: Previously seen users in CloudTrail
id: fc0edc95-ff2b-48b0-9f6f-63da3789fd03
version: 1
date: '2018-04-30'
description: This search looks for CloudTrail events where a user logs into the console,
then creates a baseline of the latest and earliest times, City, Region, and Country
we have encountered this user in our dataset, grouped by ARN, within the last 30
days.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Please validate the user name entries in `previously_seen_users_console_logins.csv`,
which is a lookup file created as a result of running this support search.
author: Jason Brewer, Splunk
search: '`cloudtrail` eventName=ConsoleLogin | rename userIdentity.arn as user | iplocation
src | eval City=if(City LIKE "",src,City),Region=if(Region LIKE "",src,Region) |
stats earliest(_time) as firstTime latest(_time) as lastTime by user src City Region
Country | outputlookup previously_seen_users_console_logins.csv | stats count'
tags:
analytics_story:
- Suspicious AWS Login Activities
detections:
- Detect AWS Console Login by User from New Country
- Detect AWS Console Login by User from New Region
- Detect AWS Console Login by User from New City
- Detect new user AWS Console Login
baselines/previously_seen_windows_service_starts.yml
-37
View File
@@ -1,37 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -30d@d
latest_time: -10m@m
search: eventtype=wineventlog_system signature_id=7036 | rename param1 as service_name | rename param2 as action | search action="running" | stats earliest(_time) as firstTime, latest(_time) as lastTime by service_name | outputlookup previously_seen_running_windows_services | stats count
creation_date: '2018-07-20'
data_metadata:
data_eventtypes:
- wineventlog_system
data_source:
- Windows Event Logs
providing_technologies:
- Microsoft Windows
description: This collects the services that have been started across your entire
enterprise.
eli5: In this support search, we look for Windows system-event code that indicates
a status change of a Windows service. In this specific log event, the `param1` field represents the
"service_name" and the `param2` represents the action/status of the service. This search will create a table of the first and last time as particular Windows service was seen to be in the `running` status.
how_to_implement: While this search does not require you to adhere to Splunk CIM,
you must be ingesting your Windows security-event logs for it to execute successfully. Please ensure that the Splunk Add-on for Microsoft Windows is version 5.0.0 or above.
id: 64ce0ade-cb01-4678-bddd-d31c0b175394
known_false_positives: ''
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
modification_date: '2020-01-13'
name: Previously Seen Running Windows Services
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '2.0'
baselines/systems_ready_for_spectre_meltdown_patch.yml
-47
View File
@@ -1,47 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: ''
earliest_time: -1d@d
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` count min(_time) as firstTime max(_time) as
lastTime FROM datamodel=Change_Analysis.All_Changes where All_Changes.object_category=registry
AND (All_Changes.object_path="HKLM\Software\Microsoft\Windows\CurrentVersion\QualityCompat*")
by All_Changes.dest, All_Changes.command, All_Changes.user, All_Changes.object,
All_Changes.object_path | `security_content_ctime(lastTime)` | `security_content_ctime(firstTime)` | `drop_dm_object_name("All_Changes")`'
creation_date: '2018-01-08'
data_metadata:
data_models:
- Change_Analysis
data_source:
- Endpoint Intel
providing_technologies:
- Carbon Black Response
- CrowdStrike Falcon
- Sysmon
- Tanium
- Ziften
description: Some AV applications can cause the Spectre/Meltdown patch for Windows
not to install successfully. This registry key is supposed to be created by the
AV engine when it has been patched to be able to handle the Windows patch. If this
key has been written, the system can then be patched for Spectre and Meltdown.
eli5: This search looks to see if a registry key was created at `HKLM\Software\Microsoft\Windows\CurrentVersion\QualityCompat`.
It will tell you when it was created and, if possible, what process created it.
how_to_implement: You need to be ingesting logs with both the process name and command-line
from your endpoints. If you are using Sysmon, you must have at least version 6.0.4
of the Sysmon TA.
id: fc0edc95-ff2b-48b0-9f6f-63da3789fd61
known_false_positives: ''
maintainers:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
modification_date: '2018-01-08'
name: Systems Ready for Spectre-Meltdown Windows Patch
original_authors:
- company: Splunk
email: davidd@splunk.com
name: David Dorsey
spec_version: 2
type: splunk
version: '1.0'
baselines/systems_ready_for_spectre_meltdown_windows_patch.yml
+23
View File
@@ -0,0 +1,23 @@
name: Systems Ready for Spectre-Meltdown Windows Patch
id: fc0edc95-ff2b-48b0-9f6f-63da3789fd61
version: 1
date: '2018-01-08'
description: Some AV applications can cause the Spectre/Meltdown patch for Windows
not to install successfully. This registry key is supposed to be created by the
AV engine when it has been patched to be able to handle the Windows patch. If this
key has been written, the system can then be patched for Spectre and Meltdown.
how_to_implement: You need to be ingesting logs with both the process name and command-line
from your endpoints. If you are using Sysmon, you must have at least version 6.0.4
of the Sysmon TA.
author: David Dorsey, Splunk
search: '| tstats `security_content_summariesonly` count min(_time) as firstTime max(_time)
as lastTime FROM datamodel=Change_Analysis.All_Changes where All_Changes.object_category=registry
AND (All_Changes.object_path="HKLM\Software\Microsoft\Windows\CurrentVersion\QualityCompat*")
by All_Changes.dest, All_Changes.command, All_Changes.user, All_Changes.object,
All_Changes.object_path | `security_content_ctime(lastTime)` | `security_content_ctime(firstTime)`
| `drop_dm_object_name("All_Changes")`'
tags:
analytics_story:
- Spectre And Meltdown Vulnerabilities
detections:
- Spectre and Meltdown Vulnerable Systems
baselines/train_ec2_excessive_runinstances.yml
-60
View File
@@ -1,60 +0,0 @@
baseline:
splunk:
macros:
- ec2_excessive_runinstances_mltk_input_filter
schedule:
cron_schedule: ''
earliest_time: -91d@d
latest_time: -1d@d
search: >-
sourcetype=aws:cloudtrail eventName=RunInstances errorCode=success `ec2_excessive_runinstances_mltk_input_filter`
| bucket span=10m _time
| stats count as instances_launched by _time src_user
| fit DensityFunction instances_launched threshold=0.0005 into ec2_excessive_runinstances_v1
creation_date: '2019-11-14'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search is used to build a Machine Learning Toolkit (MLTK) model
for how many RunInstances users do in the environment. By default, the search
uses the last 90 days of data to build the model. The model created by this search
is then used in the corresponding detection search, which identifies subsequent
outliers in the number of RunInstances performed by a user in a small time window.
eli5: Create a machine-learning (ML) model to establish a baseline for how many
RunInstances users do in the environment. This can help you identify excessive
numbers of RunInstances which may warrant further investigation to determine if there
is misuse or abuse.
how_to_implement: 'You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs.\
In addition, you must have the Machine Learning Toolkit (MLTK) version
>= 4.2 installed, along with any required dependencies. Depending on the number
of users in your environment, you may also need to adjust the value for max_inputs
in the MLTK settings for the DensityFunction algorithm, then ensure that the search
completes in a reasonable timeframe. By default, the search builds the model using
the past 30 days of data. You can modify the search window to build the model over
a longer period of time, which may give you better results. You may also want to
periodically re-run this search to rebuild the model with the latest data.\
More information on the algorithm used in the search can be found at
`https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms#DensityFunction`.'
id: fa5634df-fb05-4b4b-aba0-6115138bb1ba
known_false_positives: ''
maintainers:
- company: Splunk
email: jbrewer@splunk.com
name: Jason Brewer
modification_date: '2019-11-14'
name: Baseline of Excessive AWS Instances Launched by User - MLTK
original_authors:
- company: Splunk
email: jbrewer@splunk.com
name: Jason Brewer
spec_version: 2
type: splunk
version: '1.0'
baselines/train_ec2_excessive_terminateinstances.yml
-60
View File
@@ -1,60 +0,0 @@
baseline:
splunk:
macros:
- ec2_excessive_terminateinstances_mltk_input_filter
schedule:
cron_schedule: ''
earliest_time: -91d@d
latest_time: -1d@d
search: >-
sourcetype=aws:cloudtrail eventName=TerminateInstances errorCode=success `ec2_excessive_terminateinstances_mltk_input_filter`
| bucket span=10m _time
| stats count as instances_terminated by _time src_user
| fit DensityFunction instances_terminated threshold=0.0005 into ec2_excessive_terminateinstances_v1
creation_date: '2019-11-14'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search is used to build a Machine Learning Toolkit (MLTK) model
for how many TerminateInstances users do in the environment. By default, the search
uses the last 90 days of data to build the model. The model created by this search
is then used in the corresponding detection search, which identifies subsequent
outliers in the number of TerminateInstances performed by a user in a small time window.
eli5: Create a machine-learning (ML) model to establish a baseline for how many
TerminateInstances users do in the environment. This can help you identify excessive
numbers of TerminateInstances which may warrant further investigation to determine if there
is misuse or abuse.
how_to_implement: 'You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs.\
In addition, you must have the Machine Learning Toolkit (MLTK) version
>= 4.2 installed, along with any required dependencies. Depending on the number
of users in your environment, you may also need to adjust the value for max_inputs
in the MLTK settings for the DensityFunction algorithm, then ensure that the search
completes in a reasonable timeframe. By default, the search builds the model using
the past 30 days of data. You can modify the search window to build the model over
a longer period of time, which may give you better results. You may also want to
periodically re-run this search to rebuild the model with the latest data.\
More information on the algorithm used in the search can be found at
`https://docs.splunk.com/Documentation/MLApp/4.2.0/User/Algorithms#DensityFunction`.'
id: b28ed6de-e4ba-40f7-ae0a-93a088c774ab
known_false_positives: ''
maintainers:
- company: Splunk
email: jbrewer@splunk.com
name: Jason Brewer
modification_date: '2019-11-14'
name: Baseline of Excessive AWS Instances Terminated by User - MLTK
original_authors:
- company: Splunk
email: jbrewer@splunk.com
name: Jason Brewer
spec_version: 2
type: splunk
version: '1.0'
baselines/update_previously_seen_aws_users.yml
-50
View File
@@ -1,50 +0,0 @@
baseline:
splunk:
schedule:
cron_schedule: 0 0 1 * *
earliest_time: -60m@m
latest_time: m@m
search: sourcetype=aws:cloudtrail eventName=ConsoleLogin | rename userIdentity.arn
as user | iplocation src | eval City=if(City LIKE "",src,City),Region=if(Region
LIKE "",src,Region) | stats earliest(_time) AS firstTime latest(_time) AS lastTime
by user src City Region Country | inputlookup append=t previously_seen_users_console_logins.csv
| stats min(firstTime) as firstTime max(lastTime) as lastTime by user src City Region
Country | outputlookup previously_seen_users_console_logins.csv
creation_date: '2019-04-25'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search looks for CloudTrail events where a user logs into the console,
then updates the baseline of the latest and earliest times, City, Region, and Country
we have encountered this user in our dataset, grouped by ARN, within the last hour.
eli5: In this support search, we look for console login events by a particular user
to update the baseline cache of users/arns making the accesses, including the earliest
and latest times, City, Region, and Country a particular user ARN is seen in our
dataset, grouped by the ARN value. In cases where City and Region cannot be determined,
the source IP address is substituted for these values.
entities:
- user
- src
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Please validate the user name entries in `previously_seen_users_console_logins.csv`,
which is a lookup file created as a result of running this support search.
id: 06c036e6-d6d7-4daa-bd76-411c3d356031
known_false_positives: n/a
maintainers:
- company: Splunk
email: jbrewer@splunk.com
name: Jason Brewer
modification_date: '2018-04-30'
name: Update previously seen users in CloudTrail
original_authors:
- company: Splunk
email: jbrewer@splunk.com
name: Jason Brewer
spec_version: 2
type: splunk
version: '1.0'
baselines/update_previously_seen_users_in_cloudtrail.yml
+26
View File
@@ -0,0 +1,26 @@
name: Update previously seen users in CloudTrail
id: 06c036e6-d6d7-4daa-bd76-411c3d356031
version: 1
date: '2018-04-30'
description: This search looks for CloudTrail events where a user logs into the console,
then updates the baseline of the latest and earliest times, City, Region, and Country
we have encountered this user in our dataset, grouped by ARN, within the last hour.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. Please validate the user name entries in `previously_seen_users_console_logins.csv`,
which is a lookup file created as a result of running this support search.
author: Jason Brewer, Splunk
search: '`cloudtrail` eventName=ConsoleLogin | rename userIdentity.arn as user | iplocation
src | eval City=if(City LIKE "",src,City),Region=if(Region LIKE "",src,Region) |
stats earliest(_time) AS firstTime latest(_time) AS lastTime by user src City Region
Country | inputlookup append=t previously_seen_users_console_logins.csv | stats
min(firstTime) as firstTime max(lastTime) as lastTime by user src City Region Country
| outputlookup previously_seen_users_console_logins.csv'
tags:
analytics_story:
- Suspicious AWS Login Activities
detections:
- Detect AWS Console Login by User from New Country
- Detect AWS Console Login by User from New Region
- Detect AWS Console Login by User from New City
- Detect new user AWS Console Login
baselines/windows_updates_install_failures.yml
+13
View File
@@ -0,0 +1,13 @@
name: Windows Updates Install Failures
id: 6a4dbd1b-4502-4a11-943a-82b5ae7a42d7
version: 1
date: '2017-09-14'
description: This search is intended to give you a feel for how often Windows updates
fail to install in your environment. Fluctuations in these numbers will allow you
to determine when you should be concerned.
how_to_implement: You must be ingesting your Windows Update Logs
author: David Dorsey, Splunk
search: '| tstats `security_content_summariesonly` dc(Updates.dest) as count FROM
datamodel=Updates where Updates.vendor_product="Microsoft Windows" AND Updates.status=failure
by _time span=1d'
tags: {}
baselines/windows_updates_install_successes.yml
+13
View File
@@ -0,0 +1,13 @@
name: Windows Updates Install Successes
id: 6a80535c-86a6-4b54-894c-4b446d0c701d
version: 1
date: '2017-09-14'
description: This search is intended to give you a feel for how often successful Windows
updates are applied in your environments. Fluctuations in these numbers will allow
you to determine when you should be concerned.
how_to_implement: You must be ingesting your Windows Update Logs
author: David Dorsey, Splunk
search: '| tstats `security_content_summariesonly` dc(Updates.dest) as count FROM
datamodel=Updates where Updates.vendor_product="Microsoft Windows" AND Updates.status=installed
by _time span=1d'
tags: {}
bin/converter_v2_v3.py
+382
View File
@@ -0,0 +1,382 @@
import os
import sys
import yaml
import glob
import re
from collections import OrderedDict
from attackcti import attack_client
lift = attack_client()
all_enterprise = lift.get_enterprise(stix_format=False)
def represent_ordereddict(dumper, data):
value = []
for item_key, item_value in data.items():
node_key = dumper.represent_data(item_key)
node_value = dumper.represent_data(item_value)
value.append((node_key, node_value))
return yaml.nodes.MappingNode(u'tag:yaml.org,2002:map', value)
def attack_lookup_id(inputs_array):
outputs = []
for input in inputs_array:
for technique in all_enterprise['techniques']:
if technique['technique'].lower()==input.lower():
outputs.append(technique['external_references'][0]['external_id'])
return outputs
def remove_special_characters(input_str):
output_str = input_str.replace('.',' ').replace('/',' ').replace('(',' ').replace(')',' ').replace('&','and').replace('_',' ')
return output_str
def generate_content():
## detections ##
detection_files = glob.glob("../security-content-tmp/detections/*.yml")
story_files = glob.glob("../security-content-tmp/stories/*.yml")
old_detections = []
old_stories = []
for detection_file in detection_files:
old_detections.append(load_file(detection_file))
for story_file in story_files:
old_stories.append(load_file(story_file))
det_sto = map_detection_to_stories(old_stories)
print('## Detections ##')
for orig_dict in old_detections:
print(orig_dict['name'])
new_dict = {}
new_dict['name'] = remove_special_characters(orig_dict['name'])
new_dict['id'] = orig_dict['id']
new_dict['version'] = int(float(orig_dict['version']))
if 'modification_date' in orig_dict:
new_dict['date'] = orig_dict['modification_date']
else:
new_dict['date'] = orig_dict['creation_date']
new_dict['description'] = orig_dict['description']
new_dict['how_to_implement'] = orig_dict['how_to_implement']
new_dict['type'] = 'ESCU'
if 'references' in orig_dict:
new_dict['references'] = orig_dict['references']
for author in orig_dict['original_authors']:
author_str = author['name'] + ', ' + author['company'] + ', '
new_dict['author'] = author_str[:-2]
if 'splunk' in orig_dict['detect']:
new_dict['search'] = orig_dict['detect']['splunk']['correlation_rule']['search']
elif 'uba' in orig_dict['detect']:
new_dict['search'] = orig_dict['detect']['uba']['correlation_rule']['search']
if not str('_filter') in new_dict['search']:
new_dict['search'] = new_dict['search'] + ' | `' + new_dict['name'].replace('-','_').replace(' ','_').lower() + '_filter`'
if 'search' in new_dict:
new_dict['search'] = check_source_macro(new_dict['search'])
new_dict['search'] = change_filter_macro(new_dict)
new_dict['known_false_positives'] = orig_dict['known_false_positives']
tag_dict = {}
if orig_dict['id'] in det_sto:
tag_dict['analytics_story'] = list(det_sto[orig_dict['id']])
if 'mitre_attack' in orig_dict['mappings']:
mitre_attack_id = attack_lookup_id(orig_dict['mappings']['mitre_attack'])
if len(mitre_attack_id)>0:
tag_dict['mitre_attack_id'] = attack_lookup_id(orig_dict['mappings']['mitre_attack'])
if 'kill_chain_phases' in orig_dict['mappings']:
tag_dict['kill_chain_phases'] = orig_dict['mappings']['kill_chain_phases']
if 'cis20' in orig_dict['mappings']:
tag_dict['cis20'] = orig_dict['mappings']['cis20']
if 'nist' in orig_dict['mappings']:
tag_dict['nist'] = orig_dict['mappings']['nist']
if 'security_domain' in orig_dict:
tag_dict['security_domain'] = orig_dict['security_domain']
if 'asset_type' in orig_dict:
tag_dict['asset_type'] = orig_dict['asset_type']
new_dict['tags'] = tag_dict
ordered_new_dict = OrderedDict(new_dict.items())
new_file_name = new_dict['name'].replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
with open('detections/' + new_file_name + '.yml', 'w+' ) as outfile:
yaml.dump( new_dict , outfile , default_flow_style=False, sort_keys=False)
## baselines ##
baseline_files = glob.glob("../security-content-tmp/baselines/*.yml")
old_baselines = []
for baseline_file in baseline_files:
old_baselines.append(load_file(baseline_file))
bas_det = map_baselines_to_detection(old_detections)
old_baselines = enrich_baselines_with_stories(old_baselines, bas_det, det_sto)
bas_det_name = map_baselines_to_detection_names(old_detections)
print()
print('## Baselines ##')
for orig_dict in old_baselines:
print(orig_dict['name'])
new_dict = {}
new_dict['name'] = remove_special_characters(orig_dict['name'])
new_dict['id'] = orig_dict['id']
new_dict['version'] = int(float(orig_dict['version']))
if 'modification_date' in orig_dict:
new_dict['date'] = orig_dict['modification_date']
else:
new_dict['date'] = orig_dict['creation_date']
new_dict['description'] = orig_dict['description']
new_dict['how_to_implement'] = orig_dict['how_to_implement']
for author in orig_dict['original_authors']:
author_str = author['name'] + ', ' + author['company'] + ', '
new_dict['author'] = author_str[:-2]
new_dict['search'] = orig_dict['baseline']['splunk']['search']
new_dict['search'] = check_source_macro(new_dict['search'])
tag_dict = {}
if len(orig_dict['stories']) > 0:
tag_dict['analytics_story'] = list(orig_dict['stories'])
if orig_dict['id'] in bas_det_name:
tag_dict['detections'] = list(bas_det_name[orig_dict['id']])
new_dict['tags'] = tag_dict
ordered_new_dict = OrderedDict(new_dict.items())
new_file_name = new_dict['name'].replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
with open('baselines/' + new_file_name + '.yml', 'w+' ) as outfile:
yaml.dump( new_dict , outfile , default_flow_style=False, sort_keys=False)
## stories ##
story_files = glob.glob("../security-content-tmp/stories/*.yml")
old_stories = []
for story_file in story_files:
old_stories.append(load_file(story_file))
print()
print('## Stories ##')
for orig_dict in old_stories:
print(orig_dict['name'])
new_dict = {}
new_dict['name'] = remove_special_characters(orig_dict['name'])
new_dict['id'] = orig_dict['id']
new_dict['version'] = int(float(orig_dict['version']))
if 'modification_date' in orig_dict:
new_dict['date'] = orig_dict['modification_date']
else:
new_dict['date'] = orig_dict['creation_date']
new_dict['description'] = orig_dict['description']
new_dict['narrative'] = orig_dict['narrative']
for author in orig_dict['original_authors']:
author_str = author['name'] + ', ' + author['company'] + ', '
new_dict['author'] = author_str[:-2]
new_dict['type'] = 'ESCU'
if 'references' in orig_dict:
new_dict['references'] = orig_dict['references']
tag_dict = {}
tag_dict['analytics_story'] = remove_special_characters(orig_dict['name'])
tag_dict['usecase'] = orig_dict['usecase']
tag_dict['category'] = orig_dict['category']
new_dict['tags'] = tag_dict
ordered_new_dict = OrderedDict(new_dict.items())
new_file_name = new_dict['name'].replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
with open('stories/' + new_file_name + '.yml', 'w+' ) as outfile:
yaml.dump( new_dict , outfile , default_flow_style=False, sort_keys=False)
## response tasks ##
investigation_files = glob.glob("../security-content-tmp/investigations/*.yml")
old_investigations = []
for investigation_file in investigation_files:
old_investigations.append(load_file(investigation_file))
map_inv_det = map_investigations_to_detection(old_detections)
print()
print('## Response Tasks ##')
for orig_dict in old_investigations:
print(orig_dict['name'])
new_dict = {}
new_dict['name'] = remove_special_characters(orig_dict['name'])
new_dict['id'] = orig_dict['id']
new_dict['version'] = int(float(orig_dict['version']))
if 'modification_date' in orig_dict:
new_dict['date'] = orig_dict['modification_date']
else:
new_dict['date'] = orig_dict['creation_date']
new_dict['description'] = orig_dict['description']
new_dict['how_to_implement'] = orig_dict['how_to_implement']
for author in orig_dict['original_authors']:
author_str = author['name'] + ', ' + author['company'] + ', '
new_dict['author'] = author_str[:-2]
if 'splunk' in orig_dict['investigate']:
new_dict['inputs'] = orig_dict['investigate']['splunk']['fields_required']
new_dict['search'] = orig_dict['investigate']['splunk']['search']
new_dict = change_response_task_variable(new_dict)
# elif 'phantom' in orig_dict['investigate']:
# phantom_dict = {}
# phantom_dict['name'] = orig_dict['investigate']['phantom']['playbook_name']
# phantom_dict['url_json'] = 'todo'
# phantom_dict['url_py'] = 'todo'
# new_dict['playbook'] = phantom_dict
else:
continue
stories = get_stories_for_investigations(map_inv_det, det_sto, orig_dict)
if len(stories) > 0:
tag_dict = {}
tag_dict['analytics_story'] = stories
new_dict['tags'] = tag_dict
ordered_new_dict = OrderedDict(new_dict.items())
new_file_name = new_dict['name'].replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
with open('response_tasks/' + new_file_name + '.yml', 'w+' ) as outfile:
yaml.dump( new_dict , outfile , default_flow_style=False, sort_keys=False)
def load_file(file_path):
with open(file_path, 'r') as stream:
try:
file = list(yaml.safe_load_all(stream))[0]
except yaml.YAMLError as exc:
print(exc)
sys.exit("ERROR: reading {0}".format(file_path))
return file
def map_investigations_to_detection(detections):
inv_det = {}
for detection in detections:
if 'investigations' in detection:
for investigation in detection['investigations']:
if not (investigation['id'] in inv_det):
inv_det[investigation['id']] = {detection['id']}
else:
inv_det[investigation['id']].add(detection['id'])
return inv_det
def map_detection_to_stories(stories):
det_sto = {}
for story in stories:
if 'detections' in story:
for detection in story['detections']:
if not (detection['detection_id'] in det_sto):
det_sto[detection['detection_id']] = {remove_special_characters(story['name'])}
else:
det_sto[detection['detection_id']].add(remove_special_characters(story['name']))
return det_sto
def map_baselines_to_detection(detections):
bas_det = {}
for detection in detections:
if 'baselines' in detection:
for baseline in detection['baselines']:
if not (baseline['id'] in bas_det):
bas_det[baseline['id']] = {detection['id']}
else:
bas_det[baseline['id']].add(detection['id'])
return bas_det
def map_baselines_to_detection_names(detections):
bas_det = {}
for detection in detections:
if 'baselines' in detection:
for baseline in detection['baselines']:
if not (baseline['id'] in bas_det):
bas_det[baseline['id']] = {detection['name']}
else:
bas_det[baseline['id']].add(detection['name'])
return bas_det
def enrich_baselines_with_stories(baselines, map_bas_det, map_det_sto):
enriched_baselines = []
for baseline in baselines:
stories_set = set()
if baseline['id'] in map_bas_det:
for baseline_id in map_bas_det[baseline['id']]:
if baseline_id in map_det_sto:
stories_set = stories_set | map_det_sto[baseline_id]
baseline['stories'] = sorted(list(stories_set))
enriched_baselines.append(baseline)
return enriched_baselines
def get_stories_for_investigations(map_inv_det, map_det_sto, investigation):
story_names = set()
if investigation['id'] in map_inv_det:
detections = map_inv_det[investigation['id']]
for detection in detections:
if detection in map_det_sto:
stories = map_det_sto[detection]
story_names = story_names | stories
return sorted(list(story_names))
def check_source_macro(search):
new_search = search
mappings = {"aws:cloudtrail": "cloudtrail",
"netbackup_logs": "netbackup",
"okta_log": "okta",
"stream:http": "stream_http",
"google:gcp:pubsub:message": "google_gcp_pubsub_message",
"aws:s3:accesslogs": "aws_s3_accesslogs",
"aws:cloudwatchlogs:eks": "aws_cloudwatchlogs_eks",
"wineventlog_security": "wineventlog_security",
"XmlWinEventLog:Microsoft-Windows-Sysmon/Operational": "sysmon",
"wineventlog:microsoft-windows-wmi-activity/operational": "wmi",
"wineventlog_system": "wineventlog_system",
"aws:cloudwatchlogs:vpcflow": "cloudwatchlogs_vpcflow"}
adjust_position = 0
for match in re.finditer('(sourcetype\s?|index\s?|source\s?|eventtype\s?)=\s?([^\s)]*)',search):
if not match.group()=="source=pods" and not match.group()=="index=_internal" and not match.group()=="sourcetype=splunkd_ui_access" and not match.group()=="sourcetype=splunk_web_access":
content_match = match.group(2)
if content_match.startswith('"'):
content_match = content_match[1:]
if content_match.endswith('"'):
content_match = content_match[:-1]
new_search = new_search[0: (match.start() - adjust_position):] + new_search[(match.end() - adjust_position) + 1::]
new_search = new_search[:(match.start() - adjust_position)] + '`' + mappings[content_match] + '` ' + new_search[(match.start() - adjust_position):]
adjust_position = match.end() - match.start() - len(mappings[content_match]) - 2
#generate macro configuration
new_dict = {}
new_dict['definition'] = match.group()
new_dict['description'] = 'customer specific splunk configurations(eg- index, source, sourcetype). Replace the macro definition with configurations for your Splunk Environmnent.'
new_dict['name'] = mappings[content_match]
ordered_new_dict = OrderedDict(new_dict.items())
new_file_name = mappings[content_match]
with open('macros/' + new_file_name + '.yml', 'w+' ) as outfile:
yaml.dump( new_dict , outfile , default_flow_style=False, sort_keys=False)
return new_search
def change_filter_macro(object):
new_search = object['search']
filter_macro = re.search("([a-z0-9_]*_filter)", new_search)
if filter_macro.group(1) != (object['name'].replace(' ', '_').replace('-', '_').replace('.', '_').replace('/', '_').lower() + '_filter'):
for match in re.finditer("([a-z0-9_]*_filter)", new_search):
new_search = new_search[0: match.start() - 1:] + new_search[match.end() + 1::]
new_search = new_search[:match.start() - 1] + '`' + object['name'].replace(' ', '_').replace('-', '_').replace('.', '_').replace('/', '_').lower() + '_filter' + '` ' + new_search[match.start():]
return new_search
def change_response_task_variable(object):
if 'inputs' in object:
for input in object['inputs']:
if 'search' in object:
new_search = object['search'].replace("{" + input + "}", "$" + input + "$")
object['search'] = new_search
return object
if __name__ == "__main__":
generate_content()
bin/doc-gen.py
+163 -10
View File
@@ -3,6 +3,152 @@ import yaml
import argparse
from os import path
import sys
import re
from jinja2 import Environment, FileSystemLoader
def load_objects(file_path):
files = []
manifest_files = path.join(path.expanduser(REPO_PATH), file_path)
for file in sorted(glob.glob(manifest_files)):
files.append(load_file(file))
return files
def load_file(file_path):
with open(file_path, 'r') as stream:
try:
file = list(yaml.safe_load_all(stream))[0]
except yaml.YAMLError as exc:
print(exc)
sys.exit("ERROR: reading {0}".format(file_path))
return file
def prepare_content(stories, detections):
# enrich stories with information from detections: data_models, mitre_ids, kill_chain_phases, nists
sto_to_data_models = {}
sto_to_mitre_attack_ids = {}
sto_to_kill_chain_phases = {}
sto_to_ciss = {}
sto_to_nists = {}
sto_to_det = {}
for detection in detections:
if 'analytics_story' in detection['tags']:
for story in detection['tags']['analytics_story']:
if story in sto_to_det.keys():
sto_to_det[story].add(detection['name'])
else:
sto_to_det[story] = {detection['name']}
data_model = parse_data_models_from_search(detection['search'])
if data_model:
if story in sto_to_data_models.keys():
sto_to_data_models[story].add(data_model)
else:
sto_to_data_models[story] = {data_model}
if 'mitre_attack_id' in detection['tags']:
if story in sto_to_mitre_attack_ids.keys():
for mitre_attack_id in detection['tags']['mitre_attack_id']:
sto_to_mitre_attack_ids[story].add(mitre_attack_id)
else:
for mitre_attack_id in detection['tags']['mitre_attack_id']:
sto_to_mitre_attack_ids[story] = {mitre_attack_id}
if 'kill_chain_phases' in detection['tags']:
if story in sto_to_kill_chain_phases.keys():
for kill_chain in detection['tags']['kill_chain_phases']:
sto_to_kill_chain_phases[story].add(kill_chain)
else:
for kill_chain in detection['tags']['kill_chain_phases']:
sto_to_kill_chain_phases[story] = {kill_chain}
if 'cis20' in detection['tags']:
if story in sto_to_ciss.keys():
for cis in detection['tags']['cis20']:
sto_to_ciss[story].add(cis)
else:
for cis in detection['tags']['cis20']:
sto_to_ciss[story] = {cis}
if 'nist' in detection['tags']:
if story in sto_to_nists.keys():
for nist in detection['tags']['nist']:
sto_to_nists[story].add(nist)
else:
for nist in detection['tags']['nist']:
sto_to_nists[story] = {nist}
for story in stories:
story['detections'] = sorted(sto_to_det[story['name']])
if story['name'] in sto_to_data_models:
story['data_models'] = sorted(sto_to_data_models[story['name']])
if story['name'] in sto_to_mitre_attack_ids:
story['mitre_attack_ids'] = sorted(sto_to_mitre_attack_ids[story['name']])
if story['name'] in sto_to_kill_chain_phases:
story['kill_chain_phases'] = sorted(sto_to_kill_chain_phases[story['name']])
if story['name'] in sto_to_ciss:
story['ciss'] = sorted(sto_to_ciss[story['name']])
if story['name'] in sto_to_nists:
story['nists'] = sorted(sto_to_nists[story['name']])
#sort stories into categories
categories = []
category_names = set()
for story in stories:
if 'category' in story['tags']:
category_names.add(story['tags']['category'][0])
for category_name in sorted(category_names):
new_category = {}
new_category['name'] = category_name
new_category['stories'] = []
categories.append(new_category)
for story in stories:
for category in categories:
if category['name'] == story['tags']['category'][0]:
category['stories'].append(story)
return categories
def write_splunk_docs(stories, detections, OUTPUT_DIR):
categories = prepare_content(stories, detections)
j2_env = Environment(loader=FileSystemLoader('bin/jinja2_templates'),
trim_blocks=True)
template = j2_env.get_template('splunk_docs_categories.j2')
output_path = OUTPUT_DIR + "/splunk_docs_categories.wiki"
output = template.render(categories=categories)
with open(output_path, 'w') as f:
f.write(output)
return len(stories), output_path
def write_markdown_docs(stories, detections, OUTPUT_DIR):
categories = prepare_content(stories, detections)
j2_env = Environment(loader=FileSystemLoader('bin/jinja2_templates'),
trim_blocks=True)
template = j2_env.get_template('stories_categories.j2')
output_path = OUTPUT_DIR + "/stories_categories.md"
output = template.render(categories=categories)
with open(output_path, 'w') as f:
f.write(output)
return len(stories), output_path
# function to get unique values
@@ -314,7 +460,7 @@ def generate_stories(REPO_PATH, verbose):
return complete_stories
def write_splunk_docs(stories, detections, OUTPUT_DIR):
def write_splunk_docs_bak(stories, detections, OUTPUT_DIR):
paths = []
# Create conf files from analytics stories files
@@ -402,7 +548,7 @@ def write_splunk_docs(stories, detections, OUTPUT_DIR):
return story_count, paths
def write_markdown_docs(stories, detections, OUTPUT_DIR):
def write_markdown_docs_bak(stories, detections, OUTPUT_DIR):
paths = []
# Create conf files from analytics stories files
splunk_docs_output_path = OUTPUT_DIR + "/stories_categories.md"
@@ -510,6 +656,12 @@ def write_markdown_docs(stories, detections, OUTPUT_DIR):
return story_count, paths
def parse_data_models_from_search(search):
match = re.search('from\sdatamodel\s?=\s?([^\s.]*)',search)
if match is not None:
return match.group(1)
return False
if __name__ == "__main__":
# grab arguments
@@ -531,20 +683,21 @@ if __name__ == "__main__":
gsd = args.gen_splunk_docs
gmd = args.gen_markdown_docs
complete_stories = generate_stories(REPO_PATH, verbose)
complete_detections = generate_detections(REPO_PATH, complete_stories)
stories = load_objects("stories/*.yml")
detections = load_objects("detections/*.yml")
# complete_stories = generate_stories(REPO_PATH, verbose)
# complete_detections = generate_detections(REPO_PATH, complete_stories)
if gsd:
story_count, paths = write_splunk_docs(complete_stories, complete_detections, OUTPUT_DIR)
for p in paths:
print("{0} story documents have been successfully written to {1}".format(story_count, p))
story_count, path = write_splunk_docs(stories, detections, OUTPUT_DIR)
print("{0} story documents have been successfully written to {1}".format(story_count, path))
else:
print("--gen_splunk_docs was set to false, not generating splunk documentation")
if gmd:
story_count, paths = write_markdown_docs(complete_stories, complete_detections, OUTPUT_DIR)
for p in paths:
print("{0} story documents have been successfully written to {1}".format(story_count, p))
story_count, path = write_markdown_docs(stories, detections, OUTPUT_DIR)
print("{0} story documents have been successfully written to {1}".format(story_count, path))
else:
print("--gen_splunk_docs was set to false, not generating splunk documentation")
bin/generate.py
+351 -191
View File
@@ -11,6 +11,10 @@ from os import path
import sys
import datetime
from jinja2 import Environment, FileSystemLoader
import re
from attackcti import attack_client
import csv
# global variables
REPO_PATH = ''
@@ -18,13 +22,13 @@ VERBOSE = False
OUTPUT_PATH = ''
def load_objects(file_path):
def load_objects(file_path, VERBOSE):
files = []
manifest_files = path.join(path.expanduser(REPO_PATH), file_path)
for file in glob.glob(manifest_files):
for file in sorted(glob.glob(manifest_files)):
if VERBOSE:
print("processing manifest: {0}".format(file))
files.append(load_file(file))
return files
@@ -54,15 +58,55 @@ def generate_transforms_conf(lookups):
return output_path
def generate_savedsearches_conf(detections, investigations, baselines):
def generate_savedsearches_conf(detections, response_tasks, baselines, deployments):
for detection in detections:
# parse out data_models
data_model = parse_data_models_from_search(detection['search'])
if data_model:
detection['data_model'] = data_model
matched_deployments = get_deployments(detection, deployments)
if len(matched_deployments):
detection['deployment'] = matched_deployments[-1]
nes_fields = get_nes_fields(detection['search'], detection['deployment'])
if len(nes_fields) > 0:
detection['nes_fields'] = nes_fields
keys = ['mitre_attack', 'kill_chain_phases', 'cis20', 'nist']
mappings = {}
for key in keys:
if key == 'mitre_attack':
if 'mitre_attack_id' in detection['tags']:
mappings[key] = detection['tags']['mitre_attack_id']
else:
if key in detection['tags']:
mappings[key] = detection['tags'][key]
detection['mappings'] = mappings
for baseline in baselines:
data_model = parse_data_models_from_search(baseline['search'])
if data_model:
baseline['data_model'] = data_model
matched_deployments = get_deployments(baseline, deployments)
if len(matched_deployments):
baseline['deployment'] = matched_deployments[-1]
for response_task in response_tasks:
if 'search' in response_task:
data_model = parse_data_models_from_search(response_task['search'])
if data_model:
response_task['data_model'] = data_model
utc_time = datetime.datetime.utcnow().replace(microsecond=0).isoformat()
j2_env = Environment(loader=FileSystemLoader('bin/jinja2_templates'),
trim_blocks=True)
j2_env.filters['custom_jinja2_enrichment_filter'] = custom_jinja2_enrichment_filter
template = j2_env.get_template('savedsearches.j2')
output_path = OUTPUT_PATH + "/default/savedsearches.conf"
output = template.render(detections=detections, investigations=investigations, baselines=baselines, time=utc_time)
output = template.render(detections=detections, baselines=baselines, response_tasks=response_tasks, time=utc_time)
with open(output_path, 'w') as f:
output = output.encode('ascii', 'ignore').decode('ascii')
f.write(output)
@@ -70,7 +114,23 @@ def generate_savedsearches_conf(detections, investigations, baselines):
return output_path
def generate_analytics_story_conf(stories):
def generate_analytics_story_conf(stories, detections, response_tasks, baselines):
sto_det = map_detection_to_stories(detections)
sto_res = map_response_tasks_to_stories(response_tasks)
sto_bas = map_baselines_to_stories(baselines)
for story in stories:
if story['name'] in sto_det:
story['detections'] = list(sto_det[story['name']])
if story['name'] in sto_res:
story['response_tasks'] = list(sto_res[story['name']])
if story['name'] in sto_bas:
story['baselines'] = list(sto_bas[story['name']])
stories = prepare_stories(stories, detections)
utc_time = datetime.datetime.utcnow().replace(microsecond=0).isoformat()
@@ -85,7 +145,33 @@ def generate_analytics_story_conf(stories):
return output_path
def generate_use_case_library_conf(stories, detections, investigations, baselines):
def generate_use_case_library_conf(stories, detections, response_tasks, baselines):
sto_det = map_detection_to_stories(detections)
sto_res = map_response_tasks_to_stories(response_tasks)
for story in stories:
if story['name'] in sto_det:
story['detections'] = list(sto_det[story['name']])
if story['name'] in sto_res:
story['response_tasks'] = list(sto_res[story['name']])
story['searches'] = story['detections'] + story['response_tasks']
else:
story['searches'] = story['detections']
for detection in detections:
keys = ['mitre_attack', 'kill_chain_phases', 'cis20', 'nist']
mappings = {}
for key in keys:
if key == 'mitre_attack':
if 'mitre_attack_id' in detection['tags']:
mappings[key] = detection['tags']['mitre_attack_id']
else:
if key in detection['tags']:
mappings[key] = detection['tags'][key]
detection['mappings'] = mappings
utc_time = datetime.datetime.utcnow().replace(microsecond=0).isoformat()
@@ -94,7 +180,7 @@ def generate_use_case_library_conf(stories, detections, investigations, baseline
template = j2_env.get_template('use_case_library.j2')
output_path = OUTPUT_PATH + "/default/use_case_library.conf"
output = template.render(stories=stories, detections=detections,
investigations=investigations,
response_tasks=response_tasks,
baselines=baselines, time=utc_time)
with open(output_path, 'w') as f:
f.write(output)
@@ -102,7 +188,17 @@ def generate_use_case_library_conf(stories, detections, investigations, baseline
return output_path
def generate_macros_conf(macros):
def generate_macros_conf(macros, detections):
filter_macros = []
for detection in detections:
new_dict = {}
new_dict['definition'] = 'search *'
new_dict['description'] = 'Update this macro to limit the output results to filter out false positives. '
new_dict['name'] = detection['name']. \
replace(' ', '_').replace('-', '_').replace('.', '_').replace('/', '_').lower() + '_filter'
filter_macros.append(new_dict)
all_macros = macros + filter_macros
utc_time = datetime.datetime.utcnow().replace(microsecond=0).isoformat()
@@ -110,199 +206,263 @@ def generate_macros_conf(macros):
trim_blocks=True)
template = j2_env.get_template('macros.j2')
output_path = OUTPUT_PATH + "/default/macros.conf"
output = template.render(macros=macros, time=utc_time)
output = template.render(macros=all_macros, time=utc_time)
with open(output_path, 'w') as f:
f.write(output)
return output_path
def identify_next_steps(detections, investigations):
enriched_detections = []
for detection in detections:
if 'splunk' in detection['detect']:
if 'correlation_rule' in detection['detect']['splunk']:
investigations_output = ""
has_phantom = False
next_steps = ""
if 'investigations' in detection:
for i in detection['investigations']:
if i['type'] == 'splunk':
investigations_output += "ESCU - {0}\\n".format(i['name'])
next_steps = "{\"version\": 1, \"data\": \"Recommended following steps:\\n\\n"
next_steps += "1.[[action|escu_investigate]]: Based on ESCU investigate \
recommendations:\\ n%s\"}" % investigations_output
if i['type'] == 'phantom':
has_phantom = True
def generate_workbench_panels(response_tasks, stories):
# lets pull the playbook URL out from investigation object
playbook_url = ''
for inv in investigations:
if i['name'] == inv['name']:
playbook_url = inv['investigate']['phantom']['playbook_url']
# construct next steps with the playbook info
playbook_next_steps_string = "Splunk>Phantom Response Playbook - Monitor enrichment of the \
Splunk>Phantom Playbook called " + str(i['name']) + " and answer any \
analyst prompt in Mission Control with a response decision. \
Link to the playbook " + str(playbook_url)
next_steps = "{\"version\": 1, \"data\": \"Recommended following"
next_steps += ":\\n\\n1. [[action|runphantomplaybook]]: Phantom playbook "
next_steps += "recommendations:\\n%s\\n2. [[action|escu_investigate]]: " % (playbook_next_steps_string)
next_steps += "Based on ESCU investigate recommendations:\\n%s\"}" % (investigations_output)
if has_phantom:
detection['recommended_actions'] = 'runphantomplaybook, escu_investigate'
enriched_detections.append(detection)
sto_res = map_response_tasks_to_stories(response_tasks)
return enriched_detections
def map_investigations_to_detection(detections):
inv_det = {}
for detection in detections:
if 'investigations' in detection:
for investigation in detection['investigations']:
if not (investigation['id'] in inv_det):
inv_det[investigation['id']] = {detection['id']}
else:
inv_det[investigation['id']].add(detection['id'])
return inv_det
def map_baselines_to_detection(detections):
bas_det = {}
for detection in detections:
if 'baselines' in detection:
for baseline in detection['baselines']:
if not (baseline['id'] in bas_det):
bas_det[baseline['id']] = {detection['id']}
else:
bas_det[baseline['id']].add(detection['id'])
return bas_det
def map_detection_to_stories(stories):
det_sto = {}
for story in stories:
for detection in story['detections']:
if not (detection['detection_id'] in det_sto):
det_sto[detection['detection_id']] = {story['name']}
if story['name'] in sto_res:
response_task_names = list(sto_res[story['name']])
story['workbench_panels'] = []
for response_task_name in response_task_names:
str = 'panel://workbench_panel_' + response_task_name[7:].replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
story['workbench_panels'].append(str)
story['lowercase_name'] = story['name'].replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
workbench_panel_objects = []
for response_task in response_tasks:
if 'search' in response_task:
if 'inputs' in response_task:
response_file_name = response_task['name'].replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
response_task['lowercase_name'] = response_file_name
workbench_panel_objects.append(response_task)
j2_env = Environment(loader=FileSystemLoader('bin/jinja2_templates'),
trim_blocks=True)
template = j2_env.get_template('panel.j2')
output_path = OUTPUT_PATH + "/default/data/ui/panels/workbench_panel_" + response_file_name + ".xml"
output = template.render(search=response_task['search'])
with open(output_path, 'w') as f:
f.write(output)
j2_env = Environment(loader=FileSystemLoader('bin/jinja2_templates'),
trim_blocks=True)
template = j2_env.get_template('es_investigations.j2')
output_path = OUTPUT_PATH + "/default/es_investigations.conf"
output = template.render(response_tasks=workbench_panel_objects, stories=stories)
with open(output_path, 'w') as f:
f.write(output)
j2_env = Environment(loader=FileSystemLoader('bin/jinja2_templates'),
trim_blocks=True)
template = j2_env.get_template('workflow_actions.j2')
output_path = OUTPUT_PATH + "/default/workflow_actions.conf"
output = template.render(response_tasks=workbench_panel_objects)
with open(output_path, 'w') as f:
f.write(output)
def parse_data_models_from_search(search):
match = re.search(r'from\sdatamodel\s?=\s?([^\s.]*)', search)
if match is not None:
return match.group(1)
return False
def get_deployments(object, deployments):
matched_deployments = []
for deployment in deployments:
if 'analytics_story' in deployment['tags']:
if type(deployment['tags']['analytics_story']) is str:
tags_all_array = [deployment['tags']['analytics_story']]
else:
det_sto[detection['detection_id']].add(story['name'])
return det_sto
tags_all_array = deployment['tags']['analytics_story']
if tags_all_array[0] == 'all':
matched_deployments.append(deployment)
continue
for tag in object['tags'].keys():
if tag in deployment['tags'].keys():
if type(object['tags'][tag]) is str:
tag_array = [object['tags'][tag]]
else:
tag_array = object['tags'][tag]
for tag_value in tag_array:
if type(deployment['tags'][tag]) is str:
tag_array_deployment = [deployment['tags'][tag]]
else:
tag_array_deployment = deployment['tags'][tag]
for tag_value_deployment in tag_array_deployment:
if tag_value == tag_value_deployment:
matched_deployments.append(deployment)
continue
return matched_deployments
def enrich_investigations_with_stories(investigations, map_inv_det, map_det_sto):
enriched_investigations = []
for investigation in investigations:
stories_set = set()
if investigation['id'] in map_inv_det:
for detection_id in map_inv_det[investigation['id']]:
if detection_id in map_det_sto:
stories_set = stories_set | map_det_sto[detection_id]
def get_nes_fields(search, deployment):
nes_fields_matches = []
if 'notable' in deployment['alert_action']:
if 'nes_fields' in deployment['alert_action']['notable']:
for field in deployment['alert_action']['notable']['nes_fields']:
if (search.find(field + ' ') != -1):
nes_fields_matches.append(field)
investigation['stories'] = sorted(list(stories_set))
enriched_investigations.append(investigation)
return enriched_investigations
return nes_fields_matches
def enrich_detections_with_stories(detections, map_det_sto):
enriched_detections = []
def map_detection_to_stories(detections):
sto_det = {}
for detection in detections:
stories_set = set()
if detection['id'] in map_det_sto:
stories_set = stories_set | map_det_sto[detection['id']]
detection['stories'] = sorted(list(stories_set))
enriched_detections.append(detection)
return enriched_detections
if 'analytics_story' in detection['tags']:
for story in detection['tags']['analytics_story']:
if not (story in sto_det):
sto_det[story] = {str('ESCU - ' + detection['name'] + ' - Rule')}
else:
sto_det[story].add(str('ESCU - ' + detection['name'] + ' - Rule'))
return sto_det
def enrich_baselines_with_stories(baselines, map_bas_det, map_det_sto):
enriched_baselines = []
def map_response_tasks_to_stories(response_tasks):
sto_res = {}
for response_task in response_tasks:
if 'tags' in response_task:
if 'analytics_story' in response_task['tags']:
for story in response_task['tags']['analytics_story']:
if not (story in sto_res):
sto_res[story] = {str('ESCU - ' + response_task['name'])}
else:
sto_res[story].add(str('ESCU - ' + response_task['name']))
return sto_res
def map_baselines_to_stories(baselines):
sto_bas = {}
for baseline in baselines:
stories_set = set()
if baseline['id'] in map_bas_det:
for baseline_id in map_bas_det[baseline['id']]:
if baseline_id in map_det_sto:
stories_set = stories_set | map_det_sto[baseline_id]
if 'tags' in baseline:
if 'analytics_story' in baseline['tags']:
for story in baseline['tags']['analytics_story']:
if not (story in sto_bas):
sto_bas[story] = {str('ESCU - ' + baseline['name'])}
else:
sto_bas[story].add(str('ESCU - ' + baseline['name']))
return sto_bas
baseline['stories'] = sorted(list(stories_set))
enriched_baselines.append(baseline)
return enriched_baselines
def custom_jinja2_enrichment_filter(string, object):
customized_string = string
for key in object.keys():
customized_string = customized_string.replace("%" + key + "%", str(object[key]))
for key in object['tags'].keys():
customized_string = customized_string.replace("%" + key + "%", str(object['tags'][key]))
return customized_string
def enrich_stories(stories, detections, investigations, baselines):
enriched_stories = []
def prepare_stories(stories, detections):
# enrich stories with information from detections: data_models, mitre_ids, kill_chain_phases, nists
sto_to_data_models = {}
sto_to_mitre_attack_ids = {}
sto_to_kill_chain_phases = {}
sto_to_ciss = {}
sto_to_nists = {}
sto_to_det = {}
for detection in detections:
if 'analytics_story' in detection['tags']:
for story in detection['tags']['analytics_story']:
if story in sto_to_det.keys():
sto_to_det[story].add(str('ESCU - ' + detection['name'] + ' - Rule'))
else:
sto_to_det[story] = {str('ESCU - ' + detection['name'] + ' - Rule')}
data_model = parse_data_models_from_search(detection['search'])
if data_model:
if story in sto_to_data_models.keys():
sto_to_data_models[story].add(data_model)
else:
sto_to_data_models[story] = {data_model}
if 'mitre_attack_id' in detection['tags']:
if story in sto_to_mitre_attack_ids.keys():
for mitre_attack_id in detection['tags']['mitre_attack_id']:
sto_to_mitre_attack_ids[story].add(mitre_attack_id)
else:
for mitre_attack_id in detection['tags']['mitre_attack_id']:
sto_to_mitre_attack_ids[story] = {mitre_attack_id}
if 'kill_chain_phases' in detection['tags']:
if story in sto_to_kill_chain_phases.keys():
for kill_chain in detection['tags']['kill_chain_phases']:
sto_to_kill_chain_phases[story].add(kill_chain)
else:
for kill_chain in detection['tags']['kill_chain_phases']:
sto_to_kill_chain_phases[story] = {kill_chain}
if 'cis20' in detection['tags']:
if story in sto_to_ciss.keys():
for cis in detection['tags']['cis20']:
sto_to_ciss[story].add(cis)
else:
for cis in detection['tags']['cis20']:
sto_to_ciss[story] = {cis}
if 'nist' in detection['tags']:
if story in sto_to_nists.keys():
for nist in detection['tags']['nist']:
sto_to_nists[story].add(nist)
else:
for nist in detection['tags']['nist']:
sto_to_nists[story] = {nist}
for story in stories:
providing_technologies = set()
data_models = set()
detection_names = []
mappings = dict()
mappings["cis20"] = set()
mappings["kill_chain_phases"] = set()
mappings["mitre_attack"] = set()
mappings["nist"] = set()
mappings["mitre_technique_id"] = set()
searches = []
story['detections'] = sorted(sto_to_det[story['name']])
if story['name'] in sto_to_data_models:
story['data_models'] = sorted(sto_to_data_models[story['name']])
if story['name'] in sto_to_mitre_attack_ids:
story['mitre_attack'] = sorted(sto_to_mitre_attack_ids[story['name']])
if story['name'] in sto_to_kill_chain_phases:
story['kill_chain_phases'] = sorted(sto_to_kill_chain_phases[story['name']])
if story['name'] in sto_to_ciss:
story['cis20'] = sorted(sto_to_ciss[story['name']])
if story['name'] in sto_to_nists:
story['nist'] = sorted(sto_to_nists[story['name']])
for detection in story['detections']:
for detection_obj in detections:
if detection['detection_id'] == detection_obj['id']:
if 'providing_technologies' in detection_obj['data_metadata']:
providing_technologies = providing_technologies | set(detection_obj
['data_metadata']['providing_technologies'])
if 'data_models' in detection_obj['data_metadata']:
data_models = data_models | set(detection_obj['data_metadata']['data_models'])
if detection_obj['type'] == 'splunk':
detection_names.append("ESCU - " + detection_obj['name'] + " - Rule")
for key in detection_obj['mappings']:
mappings[key] = mappings[key] | set(detection_obj['mappings'][key])
for key in mappings.keys():
mappings[key] = sorted(list(mappings[key]))
keys = ['mitre_attack', 'kill_chain_phases', 'cis20', 'nist']
mappings = {}
for key in keys:
if key in story:
mappings[key] = story[key]
story['mappings'] = mappings
story['detection_names'] = sorted(detection_names)
searches = sorted(detection_names)
investigation_names = []
return stories
for investigation in investigations:
for s in investigation['stories']:
if s == story['name']:
if 'providing_technologies' in investigation['data_metadata']:
providing_technologies = providing_technologies | set(investigation
['data_metadata']['providing_technologies'])
if 'data_models' in investigation['data_metadata']:
data_models = data_models | set(investigation['data_metadata']['data_models'])
if investigation['type'] == 'splunk':
investigation_names.append("ESCU - " + investigation['name'])
story['investigation_names'] = sorted(investigation_names)
searches = searches + sorted(investigation_names)
def generate_mitre_lookup():
baseline_names = []
csv_mitre_rows = [["mitre_id", "technique", "tactics", "groups"]]
for baseline in baselines:
for s in baseline['stories']:
if s == story['name']:
if 'providing_technologies' in baseline['data_metadata']:
providing_technologies = providing_technologies | set(baseline['data_metadata']['providing_technologies'])
if 'data_models' in baseline['data_metadata']:
data_models = data_models | set(baseline['data_metadata']['data_models'])
if baseline['type'] == 'splunk':
baseline_names.append("ESCU - " + baseline['name'])
lift = attack_client()
all_enterprise = lift.get_enterprise(stix_format=False)
enterprise_relationships = lift.get_enterprise_relationships()
enterprise_groups = lift.get_enterprise_groups()
story['baseline_names'] = sorted(baseline_names)
searches = searches + sorted(baseline_names)
for technique in all_enterprise['techniques']:
apt_groups = []
for relationship in enterprise_relationships:
if (relationship['target_ref'] == technique['id']) and relationship['source_ref'].startswith('intrusion-set'):
for group in enterprise_groups:
if relationship['source_ref'] == group['id']:
apt_groups.append(group['name'])
story['providing_technologies'] = sorted(list(providing_technologies))
story['data_models'] = sorted(list(data_models))
story['searches'] = searches
if len(apt_groups) == 0:
apt_groups.append('no')
csv_mitre_rows.append([technique['technique_id'], technique['technique'], '|'.join(technique['tactic']).replace('-',' ').title(), '|'.join(apt_groups)])
enriched_stories.append(story)
with open('lookups/mitre_enrichment.csv', 'w', newline='') as file:
writer = csv.writer(file)
writer.writerows(csv_mitre_rows)
return enriched_stories
if __name__ == "__main__":
@@ -319,44 +479,44 @@ if __name__ == "__main__":
REPO_PATH = args.path
OUTPUT_PATH = args.output
VERBOSE = args.verbose
stories = load_objects("stories/*.yml", VERBOSE)
macros = load_objects("macros/*.yml", VERBOSE)
lookups = load_objects("lookups/*.yml", VERBOSE)
baselines = load_objects("baselines/*.yml", VERBOSE)
detections = load_objects("detections/*.yml", VERBOSE)
responses = load_objects("responses/*.yml", VERBOSE)
response_tasks = load_objects("response_tasks/*.yml", VERBOSE)
deployments = load_objects("deployments/*.yml", VERBOSE)
stories = load_objects("stories/*.yml")
macros = load_objects("macros/*.yml")
lookups = load_objects("lookups/*.yml")
detections = load_objects("detections/*.yml")
investigations = load_objects("investigations/*.yml")
baselines = load_objects("baselines/*.yml")
detections = identify_next_steps(detections, investigations)
map_inv_det = map_investigations_to_detection(detections)
map_det_sto = map_detection_to_stories(stories)
map_bas_det = map_baselines_to_detection(detections)
detections = enrich_detections_with_stories(detections, map_det_sto)
investigations = enrich_investigations_with_stories(investigations, map_inv_det, map_det_sto)
baselines = enrich_baselines_with_stories(baselines, map_bas_det, map_det_sto)
stories = enrich_stories(stories, detections, investigations, baselines)
try:
if VERBOSE:
print("generating Mitre lookups")
generate_mitre_lookup()
except:
print("WARNING: Generation of Mitre lookup failed.")
lookups_path = generate_transforms_conf(lookups)
detections = sorted(detections, key=lambda d: d['name'])
investigations = sorted(investigations, key=lambda i: i['name'])
response_tasks = sorted(response_tasks, key=lambda i: i['name'])
baselines = sorted(baselines, key=lambda b: b['name'])
detection_path = generate_savedsearches_conf(detections, investigations, baselines)
detection_path = generate_savedsearches_conf(detections, response_tasks, baselines, deployments)
stories = sorted(stories, key=lambda s: s['name'])
story_path = generate_analytics_story_conf(stories)
story_path = generate_analytics_story_conf(stories, detections, response_tasks, baselines)
use_case_lib_path = generate_use_case_library_conf(stories, detections, investigations, baselines)
use_case_lib_path = generate_use_case_library_conf(stories, detections, response_tasks, baselines)
macros = sorted(macros, key=lambda m: m['name'])
macros_path = generate_macros_conf(macros)
macros_path = generate_macros_conf(macros, detections)
generate_workbench_panels(response_tasks, stories)
if VERBOSE:
print("{0} stories have been successfully written to {1}".format(len(stories), story_path))
print("{0} stories have been successfully written to {1}".format(len(stories), use_case_lib_path))
print("{0} detections have been successfully written to {1}".format(len(detections), detection_path))
print("{0} investigations have been successfully written to {1}".format(len(investigations), detection_path))
print("{0} response tasks have been successfully written to {1}".format(len(response_tasks), detection_path))
print("{0} baselines have been successfully written to {1}".format(len(baselines), detection_path))
print("{0} macros have been successfully written to {1}".format(len(macros), macros_path))
print("security content generation completed..")
bin/jinja2_templates/analytic_stories.j2
+21 -7
View File
@@ -9,18 +9,32 @@
{% for story in stories %}
[{{ story.name }}]
category = {{ story.category[0] }}
creation_date = {{ story.creation_date }}
modification_date = {{ story.modification_date }}
category = {{ story.tags.category[0] }}
creation_date = {{ story.date }}
modification_date = {{ story.date }}
id = {{ story.id }}
version = {{ story.version }}
reference = {{ story.references | tojson }}
detection_searches = {{ story.detection_names | tojson }}
detection_searches = {{ story.detections | tojson }}
{% if story.mappings is defined %}
mappings = {{ story.mappings | tojson }}
investigative_searches = {{ story.investigation_names | tojson }}
support_searches = {{ story.baseline_names | tojson }}
{% endif %}
{% if story.response_tasks is defined %}
investigative_searches = {{ story.response_tasks | tojson }}
{% else %}
investigative_searches = []
{% endif %}
{% if story.baselines is defined %}
support_searches = {{ story.baselines | tojson }}
{% else %}
support_searches = []
{% endif %}
{% if story.data_models is defined %}
data_models = {{ story.data_models | tojson }}
providing_technologies = {{ story.providing_technologies | tojson }}
{% else %}
data_models = []
{% endif %}
providing_technologies = none
description = {{ story.description }}
{% if story.narrative is defined %}
narrative = {{ story.narrative }}
bin/jinja2_templates/es_investigations.j2
+40
View File
@@ -0,0 +1,40 @@
{% for story in stories %}
[panel_group://workbench_panel_group_{{ story.lowercase_name}}]
label = {{ story.name }}
description = {{ story.description }}
disabled = 0
panels = {{ story.workbench_panels | tojson }}
{% endfor %}
{% for response_task in response_tasks %}
[panel://workbench_panel_{{ response_task.lowercase_name }}]
label = {{ response_task.name }}
description = {{ response_task.description }}
disabled = 0
tokens = {\
{% for token in response_task.inputs %}
{% if token == 'user' %}
"user": {\
"valuePrefix": "",\
"valueSuffix": "",\
"delimiter": " OR ",\
"valueType": "primitive",\
"value": "identity",\
"default": "_1!=1"\
}{% else %}
"{{ token }}": {\
"valuePrefix": "",\
"valueSuffix": "",\
"delimiter": " OR ",\
"valueType": "primitive",\
"value": "asset",\
"default": "_1!=1"\
}{% endif %}{{ "," if not loop.last }}\
{% endfor %}
}\
{% endfor %}
bin/jinja2_templates/panel.j2
+9
View File
@@ -0,0 +1,9 @@
<panel>
<table>
<search>
<query>{{ search }}</query>
</search>
<option name="drilldown">cell</option>
<option name="wrap">false</option>
</table>
</panel>
bin/jinja2_templates/savedsearches.j2
+99 -138
View File
@@ -7,21 +7,18 @@
### ESCU DETECTIONS ###
{% for detection in detections %}
[ESCU - {{ detection.name }} - Rule]
action.escu = 0
action.escu.enabled = 1
description = {{ detection.description }}
action.escu.mappings = {{ detection.mappings | tojson }}
{% if detection.data_metadata.data_models is defined %}
action.escu.data_models = {{ detection.data_metadata.data_models | tojson }}
{% endif %}
{% if detection.eli5 is defined %}
action.escu.eli5 = {{ detection.eli5 }}
{% if detection.data_model is defined %}
action.escu.data_models = [{{ detection.data_model | tojson }}]
{% else %}
action.escu.eli5 = none
action.escu.data_models = []
{% endif %}
action.escu.eli5 = {{ detection.description }}
{% if detection.how_to_implement is defined %}
action.escu.how_to_implement = {{ detection.how_to_implement }}
{% else %}
@@ -32,67 +29,53 @@ action.escu.known_false_positives = {{ detection.known_false_positives }}
{% else %}
action.escu.known_false_positives = None
{% endif %}
action.escu.creation_date = {{ detection.creation_date }}
action.escu.modification_date = {{ detection.modification_date }}
action.escu.confidence = {{ detection.confidence }}
action.escu.creation_date = {{ detection.date }}
action.escu.modification_date = {{ detection.date }}
action.escu.confidence = high
action.escu.full_search_name = ESCU - {{ detection.name }} - Rule
action.escu.search_type = detection
{% if detection.asset_type is defined %}
action.escu.asset_at_risk = {{ detection.asset_type }}
{% if detection.providing_technologies is defined %}
action.escu.providing_technologies = {{ detection.providing_technologies | tojson }}
{% else %}
action.escu.providing_technologies = []
{% endif %}
{% if detection.entities is defined %}
action.escu.fields_required = {{ detection.entities | tojson }}
action.escu.entities = {{ detection.entities | tojson }}
{% if detection.tags.analytics_story is defined %}
action.escu.analytic_story = {{ detection.tags.analytics_story | tojson }}
{% else %}
action.escu.analytic_story = []
{% endif %}
{% if detection.data_metadata.providing_technologies is defined %}
action.escu.providing_technologies = {{ detection.data_metadata.providing_technologies | tojson }}
{% endif %}
action.escu.analytic_story = {{ detection.stories | tojson | replace("\\u0026","&") }}
{% if detection.detect.splunk is defined %}
{% set correlation_rule = detection.detect.splunk.correlation_rule %}
{% elif detection.detect.uba is defined %}
{% set correlation_rule = detection.detect.uba.correlation_rule %}
{% endif %}
{% if correlation_rule.schedule.cron_schedule is defined %}
cron_schedule = {{ correlation_rule.schedule.cron_schedule }}
{% endif %}
{% if correlation_rule.schedule.earliest_time is defined %}
dispatch.earliest_time = {{ correlation_rule.schedule.earliest_time }}
{% endif %}
{% if correlation_rule.schedule.latest_time is defined %}
dispatch.latest_time = {{ correlation_rule.schedule.latest_time }}
{% endif %}
{% if correlation_rule is defined %}
cron_schedule = {{ detection.deployment.scheduling.cron_schedule }}
dispatch.earliest_time = {{ detection.deployment.scheduling.earliest_time }}
dispatch.latest_time = {{ detection.deployment.scheduling.latest_time }}
action.correlationsearch.enabled = 1
action.correlationsearch.label = {{ detection.name }}
{% if correlation_rule.notable is defined %}
action.correlationsearch.label = ESCU - {{ detection.name }} - Rule
{% if detection.deployment.scheduling.schedule_window is defined %}
schedule_window = {{ detection.deployment.scheduling.schedule_window }}
{% endif %}
{% if detection.deployment.alert_action.notable is defined %}
action.notable = 1
{% if detection.nes_fields is defined %}
action.notable.param.nes_fields = {{ detection.nes_fields }}
{% endif %}
{% if correlation_rule.notable.nes_fields is defined %}
action.notable.param.nes_fields = {{ correlation_rule.notable.nes_fields }}
action.notable.param.rule_description = {{ correlation_rule.notable.rule_description }}
action.notable.param.rule_title = {{ correlation_rule.notable.rule_title }}
action.notable.param.security_domain = {{ detection.security_domain }}
action.notable.param.severity = {{ detection.confidence }}
action.notable.param.rule_description = {{ detection.deployment.alert_action.notable.rule_description | custom_jinja2_enrichment_filter(detection) }}
action.notable.param.rule_title = {{ detection.deployment.alert_action.notable.rule_title | custom_jinja2_enrichment_filter(detection) }}
action.notable.param.security_domain = {{ detection.tags.security_domain }}
action.notable.param.severity = high
{% endif %}
{% if (correlation_rule.notable.drilldown_name is defined) and (correlation_rule.notable.drilldown_search is defined) %}
action.notable.param.drilldown_name = {{ correlation_rule.notable.drilldown_name }}
action.notable.param.drilldown_search = {{ correlation_rule.notable.drilldown_search }}
{% if detection.deployment.alert_action.email is defined %}
action.email.subject.alert = {{ detection.deployment.alert_action.email.subject | custom_jinja2_enrichment_filter(detection) }}
action.email.to = {{ detection.deployment.alert_action.email.to }}
action.email.message.alert = {{ detection.deployment.alert_action.email.message | custom_jinja2_enrichment_filter(detection) }}
action.email.useNSSubject = 1
{% endif %}
{% if correlation_rule.risk is defined %}
{% if detection.deployment.alert_action.risk is defined %}
action.risk = 1
action.risk.param._risk_object = {{ correlation_rule.risk.risk_object }}
action.risk.param._risk_object_type = {{ correlation_rule.risk.risk_object_type[0] }}
action.risk.param._risk_score = {{ correlation_rule.risk.risk_score }}
action.risk.param._risk_object = dest
action.risk.param._risk_object_type = system
action.risk.param._risk_score = 30
action.risk.param.verbose = 0
{% endif %}
{% if correlation_rule.suppress is defined %}
alert.digest_mode = 1
alert.suppress = 1
alert.suppress.fields = {{ correlation_rule.suppress.suppress_fields }}
alert.suppress.period = {{ correlation_rule.suppress.suppress_period }}
{% endif %}
{% endif %}
action.escu.earliest_time_offset = 3600
action.escu.latest_time_offset = 86400
disabled = true
@@ -101,118 +84,96 @@ counttype = number of events
relation = greater than
quantity = 0
realtime_schedule = 0
schedule_window = auto
is_visible = false
{% if detection.detect.splunk is defined %}
search = {{ detection.detect.splunk.correlation_rule.search }}
{% elif detection.detect.uba is defined %}
search = {{ detection.detect.uba.correlation_rule.search }}
{% endif %}
search = {{ detection.search }}
{% endfor %}
### END ESCU DETECTIONS ###
### ESCU INVESTIGATIONS ###
{% for investigation in investigations %}
{% if investigation.type != 'phantom' %}
[ESCU - {{ investigation.name }}]
action.escu = 0
action.escu.enabled = 1
action.escu.search_type = investigative
action.escu.full_search_name = ESCU - {{ investigation.name }}
description = {{ investigation.description }}
action.escu.creation_date = {{ investigation.creation_date }}
action.escu.modification_date = {{ investigation.modification_date }}
action.escu.analytic_story = {{ investigation.stories | tojson | replace("\\u0026","&") }}
action.escu.earliest_time_offset = 3600
action.escu.latest_time_offset = 86400
{% if investigation.data_metadata.data_models is defined %}
action.escu.data_models = {{ investigation.data_metadata.data_models | tojson }}
{% endif %}
{% if investigation.data_metadata.providing_technologies is defined %}
action.escu.providing_technologies = {{ investigation.data_metadata.providing_technologies | tojson }}
{% endif %}
{% if investigation.eli5 is defined %}
action.escu.eli5 = {{ investigation.eli5 }}
{% else %}
action.escu.eli5 = none
{% endif %}
{% if investigation.how_to_implement is defined %}
action.escu.how_to_implement = {{ investigation.how_to_implement }}
{% else %}
action.escu.how_to_implement = none
{% endif %}
{% if investigation.known_false_positives is defined %}
action.escu.known_false_positives = {{ investigation.known_false_positives }}
{% else %}
action.escu.known_false_positives = None at this time
{% endif %}
{% if investigation.entities is defined %}
action.escu.fields_required = {{ investigation.entities | tojson }}
action.escu.entities = {{ investigation.entities | tojson }}
{% endif %}
disabled = true
schedule_window = auto
is_visible = false
search = {{ investigation.investigate.splunk.search }}
{% endif %}
{% endfor %}
### END ESCU INVESTIGATIONS ###
### ESCU BASELINES ###
{% for baseline in baselines %}
[ESCU - {{ baseline.name }}]
[ESCU - {{ baseline.name }} - Baseline]
action.escu = 0
action.escu.enabled = 1
action.escu.search_type = support
action.escu.full_search_name = ESCU - {{ baseline.name }}
description = {{ baseline.description }}
action.escu.creation_date = {{ baseline.creation_date }}
action.escu.modification_date = {{ baseline.modification_date }}
action.escu.analytic_story = {{ baseline.stories | tojson | replace("\\u0026","&") }}
{% if baseline.data_metadata.data_models is defined %}
action.escu.data_models = {{ baseline.data_metadata.data_models | tojson }}
{% endif %}
{% if baseline.baseline.splunk.schedule.earliest_time is defined %}
dispatch.earliest_time = {{ baseline.baseline.splunk.schedule.earliest_time }}
{% endif %}
{% if baseline.baseline.splunk.schedule.latest_time is defined %}
dispatch.latest_time = {{ baseline.baseline.splunk.schedule.latest_time }}
{% endif %}
{% if baseline.data_metadata.providing_technologies is defined %}
action.escu.providing_technologies = {{ baseline.data_metadata.providing_technologies | tojson }}
{% endif %}
{% if baseline.eli5 is defined %}
action.escu.eli5 = {{ baseline.eli5 }}
action.escu.creation_date = {{ baseline.date }}
action.escu.modification_date = {{ baseline.date }}
{% if baseline.tags.analytics_story is defined %}
action.escu.analytic_story = {{ baseline.tags.analytics_story | tojson }}
{% else %}
action.escu.eli5 = none
action.escu.analytic_story = []
{% endif %}
{% if baseline.data_model is defined %}
action.escu.data_models = [{{ baseline.data_model | tojson }}]
{% else %}
action.escu.data_models = []
{% endif %}
dispatch.earliest_time = {{ baseline.deployment.scheduling.earliest_time }}
dispatch.latest_time = {{ baseline.deployment.scheduling.latest_time }}
{% if baseline.deployment.scheduling.schedule_window is defined %}
schedule_window = {{ baseline.deployment.scheduling.schedule_window }}
{% endif %}
{% if baseline.providing_technologies is defined %}
action.escu.providing_technologies = {{ baseline.providing_technologies | tojson }}
{% else %}
action.escu.providing_technologies = []
{% endif %}
action.escu.eli5 = {{ baseline.description }}
{% if baseline.how_to_implement is defined %}
action.escu.how_to_implement = {{ baseline.how_to_implement }}
{% else %}
action.escu.how_to_implement = none
{% endif %}
{% if baseline.known_false_positives is defined %}
action.escu.known_false_positives = {{ baseline.known_false_positives }}
disabled = true
is_visible = false
search = {{ baseline.search }}
{% endfor %}
### ESCU RESPONSE TASKS ###
{% for response_task in response_tasks %}
{% if response_task.search is defined %}
[ESCU - {{ response_task.name }} - Response Task]
action.escu = 0
action.escu.enabled = 1
action.escu.search_type = investigative
action.escu.full_search_name = ESCU - {{ response_task.name }}
description = {{ response_task.description }}
action.escu.creation_date = {{ response_task.date }}
action.escu.modification_date = {{ response_task.date }}
{% if response_task.tags is defined %}
action.escu.analytic_story = {{ response_task.tags.analytics_story | tojson }}
{% else %}
action.escu.analytic_story = []
{% endif %}
action.escu.earliest_time_offset = 3600
action.escu.latest_time_offset = 86400
action.escu.providing_technologies = []
{% if response_task.data_model is defined %}
action.escu.data_models = [{{ response_task.data_model | tojson}}]
{% else %}
action.escu.data_models = []
{% endif %}
action.escu.eli5 = {{ response_task.description }}
action.escu.how_to_implement = none
action.escu.known_false_positives = None at this time
{% endif %}
{% if baseline.entities is defined %}
action.escu.fields_required = {{ baseline.entities | tojson }}
action.escu.entities = {{ baseline.entities | tojson }}
{% endif %}
disabled = true
schedule_window = auto
is_visible = false
search = {{ baseline.baseline.splunk.search }}
search = {{ response_task.search }}
{% endif %}
{% endfor %}
### END ESCU RESPONSE TASKS ###
### USAGE DASHBOARD CONFIGURATIONS ###
[escu-metrics-usage]
bin/jinja2_templates/splunk_docs_categories.j2
+65
View File
@@ -0,0 +1,65 @@
= Use Case Categories=
The collapse...
{% for category in categories %}
=={{ category.name }}==
{% for story in category.stories %}
==={{ story.name }}===
{{ story.description }}
<div class="toccolours mw-collapsible">
<div class="mw-collapsible-content">
====Narrative====
{{ story.narrative }}
====Detections====
{% for detection in story.detections %}
* {{ detection }}
{% endfor %}
====Data Models====
{% for data_model in story.data_models %}
* {{ data_model }}
{% endfor %}
====Tags====
=====ATT&CK=====
{% for mitre_attack_id in story.mitre_attack_ids %}
* {{ mitre_attack_id }}
{% endfor %}
=====Kill Chain Phases=====
{% for kill_chain_phase in story.kill_chain_phases %}
* {{ kill_chain_phase }}
{% endfor %}
=====CIS=====
{% for cis in story.ciss %}
* {{ cis }}
{% endfor %}
=====NIST=====
{% for nist in story.nists %}
* {{ nist }}
{% endfor %}
====References====
{% for reference in story.references %}
* {{ reference }}
{% endfor %}
date = {{ story.date }}
version = {{ story.version }}
</div>
</div>
{% endfor %}
{% endfor %}
bin/jinja2_templates/stories_categories.j2
+68
View File
@@ -0,0 +1,68 @@
# Categories
Analytics stories organized by categories
{% for category in categories %}
* [{{ category.name }}](#{{ category.name | replace(' ','-') }})
{% endfor %}
{% for category in categories %}
## {{ category.name }}
{% for story in category.stories %}
* [{{ story.name }}](#{{ story.name | replace(' ','-') }})
{% endfor %}
{% for story in category.stories %}
### {{ story.name }}
* id = {{ story.id }}
* date = {{ story.date }}
* version = {{ story.version }}
#### Description
{{ story.description }}
#### Narrative
{{ story.narrative }}
#### Detections
{% for detection in story.detections %}
* {{ detection }}
{% endfor %}
#### Data Models
{% for data_model in story.data_models %}
* {{ data_model }}
{% endfor %}
#### Mappings
##### ATT&CK
{% for mitre_attack_id in story.mitre_attack_ids %}
* {{ mitre_attack_id }}
{% endfor %}
##### Kill Chain Phases
{% for kill_chain_phase in story.kill_chain_phases %}
* {{ kill_chain_phase }}
{% endfor %}
###### CIS
{% for cis in story.ciss %}
* {{ cis }}
{% endfor %}
##### NIST
{% for nist in story.nists %}
* {{ nist }}
{% endfor %}
##### References
{% for reference in story.references %}
* {{ reference }}
{% endfor %}
{% endfor %}
{% endfor %}
bin/jinja2_templates/use_case_library.j2
+32 -25
View File
@@ -9,12 +9,12 @@
{% for story in stories %}
[analytic_story://{{ story.name }}]
category = {{ story.category[0] }}
last_updated = {{ story.modification_date }}
category = {{ story.tags.category[0] }}
last_updated = {{ story.date }}
version = {{ story.version }}
references = {{ story.references | tojson }}
maintainers = {{ story.maintainers | tojson }}
spec_version = {{ story.spec_version }}
maintainers = {{ story.author | tojson }}
spec_version = 3
searches = {{ story.searches | tojson }}
description = {{ story.description }}
{% if story.narrative is defined %}
@@ -29,48 +29,55 @@ narrative = {{ story.narrative }}
{% for detection in detections %}
[savedsearch://ESCU - {{ detection.name }} - Rule]
type = detection
asset_type = {{ detection.asset_type }}
confidence = {{ detection.confidence }}
explanation = {{ detection.eli5 }}
asset_type = {{ detection.tags.asset_type }}
confidence = medium
explanation = {{ detection.description }}
{% if detection.how_to_implement is defined %}
how_to_implement = {{ detection.how_to_implement }}
{% else %}
how_to_implement = none
{% endif %}
annotations = {{ detection.mappings | tojson }}
known_false_positives = {{ detection.known_false_positives }}
providing_technologies = {{ detection.data_metadata.providing_technologies | tojson }}
providing_technologies = []
{% endfor %}
### END DETECTIONS ###
### INVESTIGATIONS ###
### RESPONSE TASKS ###
{% for investigation in investigations %}
[savedsearch://ESCU - {{ investigation.name }}]
{% for response_task in response_tasks %}
[savedsearch://ESCU - {{ response_task.name }}]
type = investigation
explanation = none
how_to_implement = {{ investigation.how_to_implement }}
{% if investigation.known_false_positives is defined %}
known_false_positives = {{ investigation.known_false_positives }}
{% if response_task.how_to_implement is defined %}
how_to_implement = {{ response_task.how_to_implement }}
{% else %}
known_false_positives = None at this time
{% endif %}
{% if investigation.investigate.splunk is defined %}
earliest_time_offset = {{ investigation.investigate.splunk.schedule.earliest_time }}
latest_time_offset = {{ investigation.investigate.splunk.schedule.latest_time }}
{% else %}
earliest_time_offset = {{ investigation.investigate.phantom.schedule.earliest_time }}
latest_time_offset = {{ investigation.investigate.phantom.schedule.latest_time }}
how_to_implement = none
{% endif %}
known_false_positives = not defined
earliest_time_offset = 14400
latest_time_offset = 0
{% endfor %}
### END INVESTIGATIONS ###
### END RESPONSE TASKS ###
### BASELINES ###
{% for baseline in baselines %}
[savedsearch://ESCU - {{ baseline.name }}]
type = support
explanation = {{ baseline.eli5 }}
explanation = {{ baseline.description }}
{% if baseline.how_to_implement is defined %}
how_to_implement = {{ baseline.how_to_implement }}
{% else %}
how_to_implement = none
{% endif %}
{% if baseline.known_false_positives is defined %}
known_false_positives = {{ baseline.known_false_positives }}
providing_technologies = {{ baseline.data_metadata.providing_technologies | tojson }}
{% else %}
known_false_positives = not defined
{% endif %}
providing_technologies = none
{% endfor %}
### END ESCU BASELINES ###
bin/jinja2_templates/workflow_actions.j2
+18
View File
@@ -0,0 +1,18 @@
{% for response_task in response_tasks %}
{% if response_task.inputs|length == 1 %}
[workbench_panel_{{ response_task.lowercase_name }}]]
label = Workbench - {{ response_task.name }}
type = link
fields = *{{ response_task.inputs[0] }}*
display_location = field_menu
{% if response_task.inputs[0] == "user" %}
link.uri = /app/$@namespace$/ess_workbench_panel?type_identity=$@field_value$&panel=workbench_panel_{{ response_task.lowercase_name }}&drilldown_field=$@field_name$&use_drilldown_time=true
{% else %}
link.uri = /app/$@namespace$/ess_workbench_panel?type_asset=$@field_value$&panel=workbench_panel_{{ response_task.lowercase_name }}&drilldown_field=$@field_name$&use_drilldown_time=true
{% endif %}
link.target = blank
link.method = get
{% endif %}
{% endfor %}
bin/validate.py
+158 -370
View File
@@ -10,28 +10,25 @@ import jsonschema
import yaml
import sys
import argparse
import datetime
import string
import re
from os import path
def validate_object(REPO_PATH, schema_path, manifest_path, return_objects, verbose, lookups=None, macros=None):
''' Validate scheme '''
def validate_schema(REPO_PATH, type, objects):
error = False
errors = []
# uuids
baselines_uuids = []
story_uuids = []
detection_uuids = []
investigation_uuids = []
schema_file = path.join(path.expanduser(REPO_PATH), schema_path)
schema_file = path.join(path.expanduser(REPO_PATH), 'spec/' + type + '.spec.json')
try:
schema = json.loads(open(schema_file, 'rb').read())
except IOError:
print("ERROR: reading baseline schema file {0}".format(schema_file))
print("ERROR: reading schema file {0}".format(schema_file))
objects = {}
manifest_files = path.join(path.expanduser(REPO_PATH), manifest_path)
manifest_files = path.join(path.expanduser(REPO_PATH), type + '/*.yml')
for manifest_file in glob.glob(manifest_files):
if verbose:
@@ -49,365 +46,146 @@ def validate_object(REPO_PATH, schema_path, manifest_path, return_objects, verbo
try:
jsonschema.validate(instance=object, schema=schema)
except jsonschema.exceptions.ValidationError as json_ve:
print("ERROR: {0} at:\n\t{1}".format(json.dumps(json_ve.message), manifest_file))
print("\tAffected Object: {}".format(json.dumps(json_ve.instance)))
errors.append("ERROR: {0} at:\n\t{1}".format(json.dumps(json_ve.message), manifest_file))
error = True
objects[object['name']] = object
if type in objects:
objects[type].append(object)
else:
arr = []
arr.append(object)
objects[type] = arr
# validate content
if schema_path == 'spec/v2/lookups.spec.json':
error = error or validate_lookups_content(REPO_PATH, "lookups/%s", object, manifest_file)
elif schema_path == 'spec/v2/baselines.spec.json':
error = error or validate_baselines_content(object, macros, lookups, manifest_file, baselines_uuids)
elif schema_path == 'spec/v2/story.spec.json':
error = error or validate_story_content(object, manifest_file, story_uuids)
elif schema_path == 'spec/v2/detections.spec.json':
error = error or validate_detection_content(object, macros, lookups, manifest_file, detection_uuids)
elif schema_path == 'spec/v2/investigations.spec.json':
error = error or validate_investigation_content(object, macros, lookups, manifest_file, investigation_uuids)
return objects, error, errors
if return_objects:
return error, objects
def validate_objects(REPO_PATH, objects):
# uuids
uuids = []
errors = []
for lookup in objects['lookups']:
lookup_errors = validate_lookups_content(REPO_PATH, "lookups/%s", lookup)
objects_array = objects['stories'] + objects['detections'] + objects['baselines'] + objects['response_tasks'] + objects['responses']
for object in objects_array:
validation_errors, uuids = validate_standard_fields(object, uuids)
errors = errors + validation_errors
for object in objects['detections']:
errors = errors + validate_detection_search(object, objects['macros'])
for object in objects['baselines']:
errors = errors + validate_baseline_search(object, objects['macros'])
errors = lookup_errors + errors
return errors
def validate_standard_fields(object, uuids):
errors = []
if object['id'] == '':
errors.append('ERROR: Blank ID for object: %s' % object['name'])
if object['id'] in uuids:
errors.append('ERROR: Duplicate UUID found for object: %s' % object['name'])
else:
return error
uuids.append(object['id'])
# if object['name'].endswith(" "):
# errors.append(
# "ERROR: name has trailing spaces: '%s'" %
# object['name'])
invalidChars = set(string.punctuation.replace("-", ""))
if any(char in invalidChars for char in object['name']):
errors.append('ERROR: No special characters allowed in name for object: %s' % object['name'])
try:
object['description'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: description not ascii for object: %s" % object['name'])
if 'how_to_implement' in object:
try:
object['how_to_implement'].encode('ascii')
except UnicodeEncodeError:
errors.append('ERROR: how_to_implement not ascii for object: %s' % object['name'])
try:
datetime.datetime.strptime(object['date'], '%Y-%m-%d')
except ValueError:
errors.append("ERROR: Incorrect date format, should be YYYY-MM-DD for object: %s" % object['name'])
return errors, uuids
def validate_lookups_content(REPO_PATH, lookup_path, lookup, manifest_file):
error = False
def validate_detection_search(object, macros):
errors = []
if not '_filter' in object['search']:
errors.append("ERROR: Missing filter for detection: " + object['name'])
filter_macro = re.search("([a-z0-9_]*_filter)", object['search'])
if filter_macro.group(1) != (object['name'].replace(' ', '_').replace('-', '_').replace('.', '_').replace('/', '_').lower() + '_filter'):
errors.append("ERROR: filter for detection: " + object['name'] + " needs to use the name of the detection in lowercase and the special characters needs to be converted into _ .")
if any(x in object['search'] for x in ['eventtype=', 'sourcetype=', ' source=', 'index=']):
if not 'index=_internal' in object['search']:
errors.append("ERROR: Use source macro instead of eventtype, sourcetype, source or index in detection: " + object['name'])
macros_found = re.findall('\`([^\s]+)`',object['search'])
macros_filtered = []
for macro in macros_found:
if not '_filter' in macro and not 'security_content_ctime' in macro and not 'drop_dm_object_name' in macro and not 'cim_' in macro and not 'get_' in macro:
macros_filtered.append(macro)
for macro in macros_filtered:
found_macro = False
for macro_obj in macros:
if macro_obj['name'] == macro:
found_macro = True
if not found_macro:
errors.append("ERROR: macro definition for " + macro + " can't be found for detection " + object['name'])
return errors
def validate_baseline_search(object, macros):
errors = []
if any(x in object['search'] for x in ['eventtype=', 'sourcetype=', ' source=', 'index=']):
if not 'index=_internal' in object['search']:
errors.append("ERROR: Use source macro instead of eventtype, sourcetype, source or index in detection: " + object['name'])
macros_found = re.findall('\`([^\s]+)`',object['search'])
macros_filtered = []
for macro in macros_found:
if not '_filter' in macro and not 'security_content_ctime' in macro and not 'drop_dm_object_name' in macro and not 'cim_' in macro and not 'get_' in macro:
macros_filtered.append(macro)
for macro in macros_filtered:
found_macro = False
for macro_obj in macros:
if macro_obj['name'] == macro:
found_macro = True
if not found_macro:
errors.append("ERROR: macro definition for " + macro + " can't be found for detection " + object['name'])
return errors
def validate_lookups_content(REPO_PATH, lookup_path, lookup):
errors = []
if 'filename' in lookup:
lookup_csv_file = path.join(path.expanduser(REPO_PATH), lookup_path % lookup['filename'])
if not path.isfile(lookup_csv_file):
print("ERROR: filename {} does not exist".format(lookup['filename']))
print(lookup_csv_file)
print("\t{}".format(manifest_file))
error = True
return error
def validate_baselines_content(baseline, macros, lookups, baselines_manifest_file, baselines_uuids):
errors = []
error = False
baselines_errors = validate_single_baseline_content(baseline, baselines_uuids, errors, macros, lookups)
if baselines_errors:
error = True
for err in baselines_errors:
print("{0} at:\n\t {1}".format(err, baselines_manifest_file))
return error
def validate_single_baseline_content(baseline, baselines_uuids, errors, macros, lookups):
if baseline['id'] == '':
errors.append('ERROR: Blank ID')
if baseline['id'] in baselines_uuids:
errors.append('ERROR: Duplicate UUID found: %s' % baseline['id'])
else:
baselines_uuids.append(baseline['id'])
if baseline['name'].endswith(" "):
errors.append(
"ERROR: Investigation name has trailing spaces: '%s'" %
baseline['name'])
try:
baseline['description'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: description not ascii")
if 'how_to_implement' in baseline:
try:
baseline['how_to_implement'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: how_to_implement not ascii")
if 'eli5' in baseline:
try:
baseline['eli5'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: eli5 not ascii")
if 'known_false_positives' in baseline:
try:
baseline['known_false_positives'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: known_false_positives not ascii")
if 'splunk' in baseline['baseline']:
# do a regex match here instead of key values
if (baseline['baseline']['splunk']['search'].find('tstats') != -1) or \
(baseline['baseline']['splunk']['search'].find('datamodel') != -1):
if 'data_models' not in baseline['data_metadata']:
errors.append("ERROR: The Splunk search uses a data model but 'data_models' field is not set")
if not baseline['data_metadata']['data_models']:
errors.append("ERROR: The Splunk search uses a data model but 'data_models' is empty")
# do a regex match here instead of key values
if (baseline['baseline']['splunk']['search'].find('sourcetype') != -1):
if 'data_sourcetypes' not in baseline['data_metadata']:
errors.append("ERROR: The Splunk search specifies a sourcetype but 'data_sourcetypes' \
field is not set")
if not baseline['data_metadata']['data_sourcetypes']:
errors.append("ERROR: The Splunk search specifies a sourcetype but \
'data_sourcetypes' is empty")
if 'macros' in baseline['baseline']['splunk']:
for macro in baseline['baseline']['splunk']['macros']:
if macro not in macros:
errors.append("ERROR: The Splunk search specifies a macro \"{}\" \
but there is no macro manifest for it".format(macro))
if 'lookups' in baseline['baseline']['splunk']:
for lookup in baseline['baseline']['splunk']['lookups']:
if lookup not in lookups:
errors.append("ERROR: The Splunk search specifies a lookup \"{}\" \
but there is no lookup manifest for it".format(lookup))
return errors
def validate_story_content(story, story_manifest_file, story_uuids):
error = False
story_errors = validate_single_story_content(story, story_uuids)
if story_errors:
error = True
for err in story_errors:
print("{0} at:\n\t {1}".format(err, story_manifest_file))
return error
def validate_single_story_content(story, STORY_UUIDS):
''' Validate that the content of a story manifest is correct'''
errors = []
if story['id'] == '':
errors.append('ERROR: Blank ID')
if story['id'] in STORY_UUIDS:
errors.append('ERROR: Duplicate UUID found: %s' % story['id'])
else:
STORY_UUIDS.append(story['id'])
try:
story['description'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: description not ascii")
try:
story['narrative'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: narrative not ascii")
return errors
def validate_detection_content(detection, macros, lookups, manifest_file, detection_uuids):
error = False
detection_errors = validate_single_detection_content(detection, detection_uuids, macros, lookups)
if detection_errors:
error = True
for err in detection_errors:
print("{0} at:\n\t {1}".format(err, manifest_file))
return error
def validate_single_detection_content(detection, DETECTION_UUIDS, macros, lookups):
errors = []
if detection['id'] == '':
errors.append('ERROR: Blank ID')
if detection['id'] in DETECTION_UUIDS:
errors.append('ERROR: Duplicate UUID found: %s' % detection['id'])
else:
DETECTION_UUIDS.append(detection['id'])
if detection['name'].endswith(" "):
errors.append(
"ERROR: Detection name has trailing spaces: '%s'" %
detection['name'])
try:
detection['description'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: description not ascii")
if 'how_to_implement' in detection:
try:
detection['how_to_implement'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: how_to_implement not ascii")
if 'eli5' in detection:
try:
detection['eli5'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: eli5 not ascii")
if 'known_false_positives' in detection:
try:
detection['known_false_positives'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: known_false_positives not ascii")
# modded to pass validation for uba detections - not yet fleshed out
if 'splunk' in detection['detect']:
# do a regex match here instead of key values
# if (detection['detect']['splunk']['correlation_rule']['search'].find('tstats') != -1) or \
# (detection['detect']['splunk']['correlation_rule']['search'].find('datamodel') != -1):
if (detection['detect']['splunk']['correlation_rule']['search'].find('datamodel') != -1):
if 'data_models' not in detection['data_metadata']:
errors.append("ERROR: The Splunk search uses a data model but 'data_models' field is not set")
if not detection['data_metadata']['data_models']:
errors.append("ERROR: The Splunk search uses a data model but 'data_models' is empty")
# do a regex match here instead of key values
if (detection['detect']['splunk']['correlation_rule']['search'].find('sourcetype') != -1):
if 'data_sourcetypes' not in detection['data_metadata']:
errors.append("ERROR: The Splunk search specifies a sourcetype but 'data_sourcetypes' field is not set")
elif not detection['data_metadata']['data_sourcetypes']:
errors.append("ERROR: The Splunk search specifies a sourcetype but 'data_sourcetypes' is empty")
if 'macros' in detection['detect']['splunk']['correlation_rule']:
for macro in detection['detect']['splunk']['correlation_rule']['macros']:
if macro not in macros:
errors.append("ERROR: The Splunk search specifies a macro \"{}\" \
but there is no macro manifest for it".format(macro))
if 'lookups' in detection['detect']['splunk']['correlation_rule']:
for lookup in detection['detect']['splunk']['correlation_rule']['lookups']:
if lookup not in lookups:
errors.append("ERROR: The Splunk search specifies a lookup \"{}\" \
but there is no lookup manifest for it".format(lookup))
if 'notable' in detection['detect']['splunk']['correlation_rule']:
if ('drilldown_search' in detection['detect']['splunk']['correlation_rule']['notable']) ^ \
('drilldown_name' in detection['detect']['splunk']['correlation_rule']['notable']):
errors.append("ERROR: Both drilldown_search and drilldown_name must be defined")
elif 'uba' in detection['detect']:
if (detection['detect']['uba']['correlation_rule']['search'].find('tstats') != -1) or \
(detection['detect']['splunk']['correlation_rule']['search'].find('datamodel') != -1):
if 'data_models' not in detection['data_metadata']:
errors.append("ERROR: The Splunk search uses a data model but 'data_models' field is not set")
if not detection['data_metadata']['data_models']:
errors.append("ERROR: The Splunk search uses a data model but 'data_models' is empty")
# do a regex match here instead of key values
if (detection['detect']['uba']['correlation_rule']['search'].find('sourcetype') != -1):
if 'data_sourcetypes' not in detection['data_metadata']:
errors.append("ERROR: The Splunk search specifies a sourcetype but 'data_sourcetypes' \
field is not set")
if not detection['data_metadata']['data_sourcetypes']:
errors.append("ERROR: The Splunk search specifies a sourcetype but \
'data_sourcetypes' is empty")
# do a regex match here instead of key values
return errors
def validate_investigation_content(investigation, macros, lookups, manifest_file, investigation_uuids):
error = False
investigation_errors = validate_single_investigation_content(investigation, investigation_uuids, macros, lookups)
if investigation_errors:
error = True
for err in investigation_errors:
print("{0} at:\n\t {1}".format(err, manifest_file))
return error
def validate_single_investigation_content(investigation, investigation_uuids, macros, lookups):
errors = []
if investigation['id'] == '':
errors.append('ERROR: Blank ID')
if investigation['id'] in investigation_uuids:
errors.append('ERROR: Duplicate UUID found: %s' % investigation['id'])
else:
investigation_uuids.append(investigation['id'])
if investigation['name'].endswith(" "):
errors.append(
"ERROR: Investigation name has trailing spaces: '%s'" %
investigation['name'])
try:
investigation['description'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: description not ascii")
if 'how_to_implement' in investigation:
try:
investigation['how_to_implement'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: how_to_implement not ascii")
if 'eli5' in investigation:
try:
investigation['eli5'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: eli5 not ascii")
if 'known_false_positives' in investigation:
try:
investigation['known_false_positives'].encode('ascii')
except UnicodeEncodeError:
errors.append("ERROR: known_false_positives not ascii")
if 'splunk' in investigation['investigate']:
# do a regex match here instead of key values
if (investigation['investigate']['splunk']['search'].find('tstats') != -1) or \
(investigation['investigate']['splunk']['search'].find('datamodel') != -1):
if 'data_models' not in investigation['data_metadata']:
errors.append("ERROR: The Splunk search uses a data model but 'data_models' field is not set")
if not investigation['data_metadata']['data_models']:
errors.append("ERROR: The Splunk search uses a data model but 'data_models' is empty")
# do a regex match here instead of key values
if (investigation['investigate']['splunk']['search'].find('sourcetype') != -1):
if 'data_sourcetypes' not in investigation['data_metadata']:
errors.append("ERROR: The Splunk search specifies a sourcetype but 'data_sourcetypes' \
field is not set")
if not investigation['data_metadata']['data_sourcetypes']:
errors.append("ERROR: The Splunk search specifies a sourcetype but \
'data_sourcetypes' is empty")
if 'macros' in investigation['investigate']['splunk']:
for macro in investigation['investigate']['splunk']['macros']:
if macro not in macros:
errors.append("ERROR: The Splunk search specifies a macro \"{}\" \
but there is no macro manifest for it".format(macro))
if 'lookups' in investigation['investigate']['splunk']:
for lookup in investigation['investigate']['splunk']['lookups']:
if lookup not in lookups:
errors.append("ERROR: The Splunk search specifies a lookup \"{}\" \
but there is no lookup manifest for it".format(lookup))
errors.append("ERROR: filename {} does not exist".format(lookup['filename']))
return errors
@@ -424,16 +202,26 @@ if __name__ == "__main__":
REPO_PATH = args.path
verbose = args.verbose
macros_error, macros = validate_object(REPO_PATH, 'spec/v2/macros.spec.json', 'macros/*.yml', True, verbose)
lookups_error, lookups = validate_object(REPO_PATH, 'spec/v2/lookups.spec.json', 'lookups/*.yml', True, verbose)
story_error = validate_object(REPO_PATH, 'spec/v2/story.spec.json', 'stories/*.yml', False, verbose)
detection_error = validate_object(REPO_PATH, 'spec/v2/detections.spec.json', 'detections/*.yml', False,
verbose, lookups, macros)
investigation_error = validate_object(REPO_PATH, 'spec/v2/investigations.spec.json', 'investigations/*.yml',
False, verbose, lookups, macros)
baseline_error = validate_object(REPO_PATH, 'spec/v2/baselines.spec.json', 'baselines/*.yml', False, verbose, lookups, macros)
validation_objects = ['macros','lookups','stories','detections','baselines','response_tasks','responses','deployments']
if story_error or detection_error or investigation_error or baseline_error or macros_error or lookups_error:
objects = {}
schema_error = False
schema_errors = []
for validation_object in validation_objects:
objects, error, errors = validate_schema(REPO_PATH, validation_object, objects)
schema_error = schema_error or error
if len(errors) > 0:
schema_errors = schema_errors + errors
validation_errors = validate_objects(REPO_PATH, objects)
schema_errors = schema_errors + validation_errors
for schema_error in schema_errors:
print(schema_error)
if schema_error or len(schema_errors) > 0:
sys.exit("Errors found")
else:
print("No Errors found")
dashboards/.gitkeep
View File
deployments/enterprise_security_deployment_configuration.yml
+20
View File
@@ -0,0 +1,20 @@
name: Enterprise Security deployment configuration
id: bc91a8cd-35e7-4bb2-6140-e756cc46f212
date: '2020-04-27'
description: This configuration file applies to all correlation searches that are used for detection
author: Bhavin Patel
scheduling:
cron_schedule: '*/30 * * * *'
earliest_time: -30m
latest_time: now
schedule_window: auto
alert_action:
notable:
rule_description: '%description%'
rule_title: '%name%'
nes_fields:
- user
- dest
- src
tags:
analytics_story: all
detections/abnormally_high_aws_instances_launched_by_user.yml
+37
View File
@@ -0,0 +1,37 @@
name: Abnormally High AWS Instances Launched by User
id: 2a9b80d3-6340-4345-b5ad-290bf5d0dac4
version: 1
date: '2018-02-26'
description: This search looks for CloudTrail events where a user successfully launches
an abnormally high number of instances.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. The threshold value should be tuned to your environment.
type: ESCU
references: []
author: Bhavin Patel, Splunk
search: '`cloudtrail` eventName=RunInstances errorCode=success | bucket span=10m _time
| stats count AS instances_launched by _time userName | eventstats avg(instances_launched)
as total_launched_avg, stdev(instances_launched) as total_launched_stdev | eval
threshold_value = 4 | eval isOutlier=if(instances_launched > total_launched_avg+(total_launched_stdev
* threshold_value), 1, 0) | search isOutlier=1 AND _time >= relative_time(now(),
"-10m@m") | eval num_standard_deviations_away = round(abs(instances_launched - total_launched_avg)
/ total_launched_stdev, 2) | table _time, userName, instances_launched, num_standard_deviations_away,
total_launched_avg, total_launched_stdev | `abnormally_high_aws_instances_launched_by_user_filter`'
known_false_positives: Many service accounts configured within an AWS infrastructure
are known to exhibit this behavior. Please adjust the threshold values and filter
out service accounts from the output. Always verify if this search alerted on a
human user.
tags:
analytics_story:
- AWS Cryptomining
- Suspicious AWS EC2 Activities
kill_chain_phases:
- Actions on Objectives
cis20:
- CIS 13
nist:
- DE.DP
- DE.AE
security_domain: network
asset_type: AWS Instance
detections/abnormally_high_aws_instances_launched_by_user___mltk.yml
+33
View File
@@ -0,0 +1,33 @@
name: Abnormally High AWS Instances Launched by User - MLTK
id: dec41ad5-d579-42cb-b4c6-f5dbb778bbe5
version: 1
date: '2019-11-14'
description: This search looks for CloudTrail events where a user successfully launches
an abnormally high number of instances.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. The threshold value should be tuned to your environment.
type: ESCU
references: []
author: Jason Brewer, Splunk
search: '`cloudtrail` eventName=RunInstances errorCode=success `abnormally_high_aws_instances_launched_by_user___mltk_filter`
| bucket span=10m _time | stats count as instances_launched by _time src_user |
apply ec2_excessive_runinstances_v1 | rename "IsOutlier(instances_launched)" as
isOutlier | where isOutlier=1'
known_false_positives: Many service accounts configured within an AWS infrastructure
are known to exhibit this behavior. Please adjust the threshold values and filter
out service accounts from the output. Always verify if this search alerted on a
human user.
tags:
analytics_story:
- Cloud Cryptomining
- Suspicious AWS EC2 Activities
kill_chain_phases:
- Actions on Objectives
cis20:
- CIS 13
nist:
- DE.DP
- DE.AE
security_domain: network
asset_type: AWS Instance
detections/abnormally_high_aws_instances_terminated_by_user.yml
+37
View File
@@ -0,0 +1,37 @@
name: Abnormally High AWS Instances Terminated by User
id: ada0f478-84a8-4641-s3f3-d82362dffd75
version: 1
date: '2018-02-26'
description: This search looks for CloudTrail events where an abnormally high number
of instances were successfully terminated by a user in a 10-minute window
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs.
type: ESCU
references: []
author: Bhavin Patel, Splunk
search: '`cloudtrail` eventName=TerminateInstances errorCode=success | bucket span=10m
_time | stats count AS instances_terminated by _time userName | eventstats avg(instances_terminated)
as total_terminations_avg, stdev(instances_terminated) as total_terminations_stdev
| eval threshold_value = 4 | eval isOutlier=if(instances_terminated > total_terminations_avg+(total_terminations_stdev
* threshold_value), 1, 0) | search isOutlier=1 AND _time >= relative_time(now(),
"-10m@m")| eval num_standard_deviations_away = round(abs(instances_terminated -
total_terminations_avg) / total_terminations_stdev, 2) |table _time, userName, instances_terminated,
num_standard_deviations_away, total_terminations_avg, total_terminations_stdev |
`abnormally_high_aws_instances_terminated_by_user_filter`'
known_false_positives: Many service accounts configured with your AWS infrastructure
are known to exhibit this behavior. Please adjust the threshold values and filter
out service accounts from the output. Always verify whether this search alerted
on a human user.
tags:
analytics_story:
- Suspicious AWS EC2 Activities
kill_chain_phases:
- Actions on Objectives
cis20:
- CIS 13
nist:
- DE.DP
- DE.AE
security_domain: network
asset_type: AWS Instance
detections/abnormally_high_aws_instances_terminated_by_user___mltk.yml
+32
View File
@@ -0,0 +1,32 @@
name: Abnormally High AWS Instances Terminated by User - MLTK
id: 1c02b86a-cd85-473e-a50b-014a9ac8fe3e
version: 1
date: '2019-11-14'
description: This search looks for CloudTrail events where a user successfully terminates
an abnormally high number of instances.
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. The threshold value should be tuned to your environment.
type: ESCU
references: []
author: Jason Brewer, Splunk
search: '`cloudtrail` eventName=TerminateInstances errorCode=success `abnormally_high_aws_instances_terminated_by_user___mltk_filter`
| bucket span=10m _time | stats count as instances_terminated by _time src_user |
apply ec2_excessive_terminateinstances_v1 | rename "IsOutlier(instances_terminated)"
as isOutlier | where isOutlier=1'
known_false_positives: Many service accounts configured within an AWS infrastructure
are known to exhibit this behavior. Please adjust the threshold values and filter
out service accounts from the output. Always verify if this search alerted on a
human user.
tags:
analytics_story:
- Suspicious AWS EC2 Activities
kill_chain_phases:
- Actions on Objectives
cis20:
- CIS 13
nist:
- DE.DP
- DE.AE
security_domain: network
asset_type: AWS Instance
detections/abnormally_high_ec2_instances_launched.yml
-108
View File
@@ -1,108 +0,0 @@
asset_type: AWS Instance
confidence: medium
creation_date: '2018-02-26'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search looks for CloudTrail events where a user successfully launches
an abnormally high number of instances.
detect:
splunk:
correlation_rule:
notable:
nes_fields: userName
rule_description: An abnormally high number of instances were launched by
a user within in a 10-minute window
rule_title: High Number of instances launched by $userName$
risk:
risk_object: userName
risk_object_type:
- user
risk_score: 30
schedule:
cron_schedule: '*/10 * * * *'
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=RunInstances errorCode=success |
bucket span=10m _time | stats count AS instances_launched by _time userName
| eventstats avg(instances_launched) as total_launched_avg, stdev(instances_launched)
as total_launched_stdev | eval threshold_value = 4 | eval isOutlier=if(instances_launched
> total_launched_avg+(total_launched_stdev * threshold_value), 1, 0) | search
isOutlier=1 AND _time >= relative_time(now(), "-10m@m") | eval num_standard_deviations_away
= round(abs(instances_launched - total_launched_avg) / total_launched_stdev,
2) | table _time, userName, instances_launched, num_standard_deviations_away,
total_launched_avg, total_launched_stdev
suppress:
suppress_fields: userName
suppress_period: 3600s
eli5: In this search, we query CloudTrail logs to look for events where an instance
is successfully launched by a particular user. Since we want to detect a high number
of instances launched within a short period, we create event buckets for 10-minute
windows. We then calculate the total number of instances launched by a particular
user, as well as the average and standard deviation values. Assign a `threshold_value`
in the search. Start with 3 (but it will likely need to be tweaked for your environment).
The `eval` function will set the outlier 1 if the number of instances is greater
than the average number of instances terminated, added to the multiplied value of
threshold and standard deviation. For your reference, we then keep only the outliers
and calculate the number of standard deviations away the value is from the average.
entities:
- userName
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. The threshold value should be tuned to your environment.
id: 2a9b80d3-6340-4345-b5ad-290bf5d0dac4
investigations:
- id: bc91a8cd-35e7-4bb2-6140-e756cc46fd71
name: AWS Investigate User Activities By ARN
type: splunk
- id: 0e40fe83-3edb-4d86-8206-8fed36529ca6
name: Get EC2 Launch Details
type: splunk
- id: f3db4d1b-5f33-4b01-c541-c7ah9514c242
name: Get EC2 Instance Details by instanceId
type: splunk
- id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
name: Get Notable History
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
name: Get Notable Info
type: splunk
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd74
name: Get User Information from Identity Table
type: splunk
- id: bc91a8cd-35e7-4bb2-6140-e756cc46fd11
name: Investigate AWS activities via region name
type: splunk
known_false_positives: Many service accounts configured within an AWS infrastructure
are known to exhibit this behavior. Please adjust the threshold values and filter
out service accounts from the output. Always verify if this search alerted on a
human user.
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
mappings:
cis20:
- CIS 13
kill_chain_phases:
- Actions on Objectives
mitre_attack:
- Execution
nist:
- DE.DP
- DE.AE
modification_date: '2018-02-26'
name: Abnormally High AWS Instances Launched by User
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
references: []
security_domain: network
spec_version: 2
type: splunk
version: '1.0'
detections/abnormally_high_instance_termination.yml
-106
View File
@@ -1,106 +0,0 @@
asset_type: AWS Instance
confidence: medium
creation_date: '2018-02-26'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search looks for CloudTrail events where an abnormally high number
of instances were successfully terminated by a user in a 10-minute window
detect:
splunk:
correlation_rule:
notable:
nes_fields: userName
rule_description: An abnormally high number of instances were terminated by
a user in a 10-minute window
rule_title: High number of instances terminated by $userName$
risk:
risk_object: userName
risk_object_type:
- user
risk_score: 30
schedule:
cron_schedule: '*/10 * * * *'
earliest_time: -30d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=TerminateInstances errorCode=success
| bucket span=10m _time | stats count AS instances_terminated by _time userName
| eventstats avg(instances_terminated) as total_terminations_avg, stdev(instances_terminated)
as total_terminations_stdev | eval threshold_value = 4 | eval isOutlier=if(instances_terminated
> total_terminations_avg+(total_terminations_stdev * threshold_value), 1,
0) | search isOutlier=1 AND _time >= relative_time(now(), "-10m@m")| eval
num_standard_deviations_away = round(abs(instances_terminated - total_terminations_avg)
/ total_terminations_stdev, 2) |table _time, userName, instances_terminated,
num_standard_deviations_away, total_terminations_avg, total_terminations_stdev
suppress:
suppress_fields: userName
suppress_period: 3600s
eli5: In this search, we query CloudTrail logs to look for events where an instance
is successfully terminated by a particular user. Since we want to detect a high
number of instances terminated within a short period, we create event buckets for
10-minute windows. We then calculate the total number of instances terminated by
a particular user, as well as the average- and standard-deviation values. Assign
a `threshold_value` in the search. Try starting with 3 (but it will likely need
to be tweaked for your environment). The `eval` function will set the outlier to
1 if the number of instances is greater than the average number of instances terminated,
added to the multiplied value of threshold and standard deviation. We then filter
out outliers with a value of 1 and show only those instance-termination events that
happened within the previous 10 minutes.
entities:
- userName
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs.
id: ada0f478-84a8-4641-s3f3-d82362dffd75
investigations:
- id: bc91a8cd-35e7-4bb2-6140-e756cc46fd71
name: AWS Investigate User Activities By ARN
type: splunk
- id: f3db4d1b-5f33-4b01-c541-c7ah9514c242
name: Get EC2 Instance Details by instanceId
type: splunk
- id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
name: Get Notable History
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
name: Get Notable Info
type: splunk
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd74
name: Get User Information from Identity Table
type: splunk
- id: bc91a8cd-35e7-4bb2-6140-e756cc46fd11
name: Investigate AWS activities via region name
type: splunk
known_false_positives: Many service accounts configured with your AWS infrastructure
are known to exhibit this behavior. Please adjust the threshold values and filter
out service accounts from the output. Always verify whether this search alerted
on a human user.
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
mappings:
cis20:
- CIS 13
kill_chain_phases:
- Actions on Objectives
mitre_attack:
- Execution
nist:
- DE.DP
- DE.AE
modification_date: '2018-02-26'
name: Abnormally High AWS Instances Terminated by User
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
references: []
security_domain: network
spec_version: 2
type: splunk
version: '1.0'
detections/access_lsass_memory_for_dump_creation.yml
+33 -86
View File
@@ -1,88 +1,35 @@
asset_type: Windows
confidence: high
creation_date: '2019-12-06'
data_metadata:
data_eventtypes:
- wineventlog_sysmon
data_source:
- Windows Event Logs
providing_technologies:
- Microsoft Windows
description: Detect memory dumping of the LSASS process.
detect:
splunk:
correlation_rule:
notable:
nes_fields: dest
rule_description: Possible attempt at credential dumping was detected on $dest$.
rule_title: LSASS memory dump detected on $dest$.
risk:
risk_object: dest
risk_object_type:
- system
risk_score: 70
macros:
- sysmon
- access_lsass_memory_for_dump_creation_filter
schedule:
cron_schedule: 0 * * * *
earliest_time: -70m@m
latest_time: -10m@m
search: '`sysmon` EventCode=10 TargetImage=*lsass.exe CallTrace=*dbgcore.dll* OR CallTrace=*dbghelp.dll*
| stats count min(_time) as firstTime max(_time) as lastTime by Computer, TargetImage, TargetProcessId, SourceImage, SourceProcessId
| rename Computer as dest
| `security_content_ctime(firstTime)`| `security_content_ctime(lastTime)`
| `access_lsass_memory_for_dump_creation_filter` '
suppress:
suppress_fields: dest
suppress_period: 86400s
eli5: dbgcore.dll is a specifc DLL for Windows core debugging. It is used to obtain a memory dump of a process.
This search detects the usage of this DLL for creating a memory dump of LSASS process. Memory dumps of the LSASS process
can be created with tools such as Windows Task Manager or procdump.
entities:
- dest
how_to_implement: This search requires Sysmon Logs and a Sysmon configuration, which includes EventCode 10 for lsass.exe. This search uses an input macro named `sysmon`. We strongly recommend that you specify your environment-specific configurations (index, source, sourcetype, etc.) for Windows Sysmon logs. Replace the macro definition with configurations for your Splunk environment. The search also uses a post-filter macro designed to filter out known false positives.
id: fb4c31b0-13e8-4155-8aa5-24de4b8d6717
investigations:
- id: 5de385bf-4f1e-404e-9b67-92d162ff8938ad
name: Investigate Previous Unseen User
type: splunk
- id: 097e8030-8662-4254-a735-bf0bdda696e3
name: Investigate Failed Logins for Multiple Destinations
type: splunk
- id: ed3fff45-cba6-4990-983f-6fac72bee659
name: Investigate Pass the Hash Attempts
type: splunk
- id: 990007ad-d798-4b29-ab2f-f0034144c937
name: Investigate Pass the Ticket Attempts
type: splunk
known_false_positives: Administrators can create memory dumps for debugging purposes, but memory dumps of the LSASS process would be unusual.
maintainers:
- company: Splunk
email: pbareiss@splunk.com
name: Patrick Bareiss
mappings:
cis20:
- CIS 6
- CIS 8
kill_chain_phases:
- Actions on Objectives
mitre_attack:
- Credential Access
- Credential Dumping
mitre_technique_id:
- T1003
nist:
- DE.CM
modification_date: '2019-12-06'
name: Access LSASS Memory for Dump Creation
original_authors:
- company: Splunk
email: pbareiss@splunk.com
name: Patrick Bareiss
id: fb4c31b0-13e8-4155-8aa5-24de4b8d6717
version: 2
date: '2019-12-06'
description: Detect memory dumping of the LSASS process.
how_to_implement: This search requires Sysmon Logs and a Sysmon configuration, which
includes EventCode 10 for lsass.exe. This search uses an input macro named `sysmon`.
We strongly recommend that you specify your environment-specific configurations
(index, source, sourcetype, etc.) for Windows Sysmon logs. Replace the macro definition
with configurations for your Splunk environment. The search also uses a post-filter
macro designed to filter out known false positives.
type: ESCU
references:
- https://2017.zeronights.org/wp-content/uploads/materials/ZN17_Kheirkhabarov_Hunting_for_Credentials_Dumping_in_Windows_Environment.pdf
security_domain: endpoint
spec_version: 2
type: splunk
version: "2"
- https://2017.zeronights.org/wp-content/uploads/materials/ZN17_Kheirkhabarov_Hunting_for_Credentials_Dumping_in_Windows_Environment.pdf
author: Patrick Bareiss, Splunk
search: '`sysmon` EventCode=10 TargetImage=*lsass.exe CallTrace=*dbgcore.dll* OR CallTrace=*dbghelp.dll*
| stats count min(_time) as firstTime max(_time) as lastTime by Computer, TargetImage,
TargetProcessId, SourceImage, SourceProcessId | rename Computer as dest | `security_content_ctime(firstTime)`|
`security_content_ctime(lastTime)` | `access_lsass_memory_for_dump_creation_filter` '
known_false_positives: Administrators can create memory dumps for debugging purposes,
but memory dumps of the LSASS process would be unusual.
tags:
analytics_story:
- Credential Dumping
mitre_attack_id:
- T1003
kill_chain_phases:
- Actions on Objectives
cis20:
- CIS 6
- CIS 8
nist:
- DE.CM
security_domain: endpoint
asset_type: Windows
detections/account_harvesting.yml
-105
View File
@@ -1,105 +0,0 @@
asset_type: Account
confidence: medium
creation_date: '2018-07-12'
data_metadata:
data_source:
- Network Communications
data_sourcetypes:
- stream:http
providing_technologies:
- Splunk Stream
description: This search is used to identify the creation of multiple user accounts
using the same email domain name.
detect:
splunk:
correlation_rule:
notable:
nes_fields: src_user
rule_description: This search is used to identify multiple created accounts
tied to a specific email domain. Such activity is often indicative of account
harvesting. A list of $src_user$ accounts were created.
rule_title: 'Web Fraud Detection: Possible Account Harvesting'
risk:
risk_object: src_user
risk_object_type:
- user
risk_score: 40
schedule:
cron_schedule: 0 1 * * *
earliest_time: -1445m@m
latest_time: -10m@m
search: sourcetype=stream:http http_content_type=text* uri="/magento2/customer/account/loginPost/"
| rex field=cookie "form_key=(?<SessionID>\w+)" | rex field=form_data "login\[username\]=(?<Username>[^&|^$]+)"
| search Username=* | rex field=Username "@(?<email_domain>.*)"|stats dc(Username)
as UniqueUsernames list(Username) as src_user by email_domain|where UniqueUsernames>
25
suppress:
suppress_fields: user
suppress_period: 3600s
eli5: When a fraudster is setting the stage for a campaign, they will often create
many user accounts on the website. This is a simple example of how to detect a many-account
creation hosted on a Magento2 e-commerce platform, where the fraudster is using
email addresses from a single email domain.
entities:
- src_user
how_to_implement: We start with a dataset that provides visibility into the email
address used for the account creation. In this example, we are narrowing our search
down to the single web page that hosts the Magento2 e-commerce platform (via URI)
used for account creation, the single http content-type to grab only the user's
clicks, and the http field that provides the username (form_data), for performance
reasons. After we have the username and email domain, we look for numerous account
creations per email domain. Common data sources used for this detection are customized
Apache logs or Splunk Stream.
id: 31337aaa-941d-4ada-81ac-q2a17be5bf0d
investigations:
- id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
name: Get Notable History
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
name: Get Notable Info
type: splunk
- id: bc91a8cf-35e7-4bb2-1120-e756cc06fd89
name: Get Web Session Information via session_id
type: splunk
- id: 5df39b3f-447d-4869-b673-8f45ad4616fe
name: Get Emails From Specific Sender
type: splunk
known_false_positives: As is common with many fraud-related searches, we are usually
looking to attribute risk or synthesize relevant context with loosely written detections
that simply detect anamolous behavior. This search will need to be customized to
fit your environment&#151;improving its fidelity by counting based on something
much more specific, such as a device ID that may be present in your dataset. Consideration
for whether the large number of registrations are occuring from a first-time seen
domain may also be important. Extending the search window to look further back
in time, or even calculating the average per hour/day for each email domain to look
for an anomalous spikes, will improve this search. You can also use Shannon entropy
or Levenshtein Distance (both courtesy of URL Toolbox) to consider the randomness
or similarity of the email name or email domain, as the names are often machine-generated.
maintainers:
- company: Splunk
email: Mayhem@splunk.com
name: Jim Apger
mappings:
cis20:
- CIS 16
kill_chain_phases:
- Actions on Objectives
mitre_attack:
- Persistence
- Create Account
nist:
- DE.CM
- DE.DP
modification_date: '2018-10-08'
name: Web Fraud - Account Harvesting
original_authors:
- company: Splunk
email: Mayhem@splunk.com
name: Jim Apger
references:
- https://splunkbase.splunk.com/app/2734/
- https://splunkbase.splunk.com/app/1809/
security_domain: threat
spec_version: 2
type: splunk
version: '1.0'
detections/acl_with_open_ports.yml
-95
View File
@@ -1,95 +0,0 @@
asset_type: AWS Instance
confidence: medium
creation_date: '2017-01-08'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: The search looks for CloudTrail events to detect if any network ACLs
were created with all the ports open to a specified CIDR.
detect:
splunk:
correlation_rule:
notable:
nes_fields: aws_account_id, src, arn
rule_description: $src_user$ created a network access control list with all
ports open.
rule_title: Network ACL created with all ports open by $src_user$
risk:
risk_object: arn
risk_object_type:
- user
risk_score: 30
schedule:
cron_schedule: 0 * * * *
earliest_time: -1d@d
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventName=CreateNetworkAclEntry | mvexpand
requestParameters | mvexpand responseElements | search requestParameters.portRange.from=1024
requestParameters.portRange.to=65535 requestParameters.ruleAction=allow |
rename userIdentity.arn as arn | rename requestParameters.networkAclId as
networkAclId | table _time aws_account_id src userName arn networkAclId requestParameters.*
responseElements.*
suppress:
suppress_fields: arn
suppress_period: 14400s
eli5: A network access control list (ACL) is a layer of security for your VPC that
acts as a firewall for controlling traffic in and out of one or more subnets. Network
ACLs with all open ports have a larger attack surface. This search looks for events
within your CloudTrail logs to check if there were any Network ACLs created with
ports ranging from 1024 to 65525. This search will create a table comprised of AWS
account id, src, user and all parameters of the request made by the user and the
server response.
entities:
- src_user
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS, version 4.4.0 or later, and configure your CloudTrail
inputs.
id: ada0f478-84a8-4641-a3f1-d82362d6bd75
investigations:
- id: bc91a8cd-35e7-4bb2-6140-e756cc46fd71
name: AWS Investigate User Activities By ARN
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7ah9534c242
name: AWS Network ACL Details from ID
type: splunk
- id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
name: Get Notable History
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
name: Get Notable Info
type: splunk
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd74
name: Get User Information from Identity Table
type: splunk
known_false_positives: It's possible that an admin has created this ACL with all ports
open for some legitimate purpose however, this should be scoped and not allowed
in production environment.
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
mappings:
cis20:
- CIS 11
kill_chain_phases:
- Actions on Objectives
mitre_attack:
- Persistence
nist:
- DE.DP
- DE.AE
modification_date: '2017-01-10'
name: AWS Network Access Control List Created with All Open Ports
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
references: []
security_domain: network
spec_version: 2
type: splunk
version: '1.0'
detections/activity_related_to_pass_the_hash.yml
-109
View File
@@ -1,109 +0,0 @@
asset_type: Endpoint
confidence: high
creation_date: '2016-09-13'
data_metadata:
data_eventtypes:
- wineventlog_security
data_source:
- Authentication
providing_technologies:
- Microsoft Windows
description: This search looks for specific authentication events from the Windows
Security Event logs to detect potential attempts at using the Pass-the-Hash technique.
detect:
splunk:
correlation_rule:
notable:
nes_fields: src_ip, dest, user
rule_description: This search looks for Authentication log events from the
Windows Security Audit logs to detect potential attempts for Passing the
Hash
rule_title: Detect Activity Related to Pass the Hash
risk:
risk_object: dest
risk_object_type:
- system
risk_score: 10
macros:
- wineventlog_security
- detect_activity_related_to_pass_the_hash_attacks_filter
schedule:
cron_schedule: 0 * * * *
earliest_time: -70m@m
latest_time: -10m@m
search: '`wineventlog_security` EventCode=4624 (Logon_Type=3 LogonProcessName=NtLmSsp WorkstationName=WORKSTATION NOT AccountName="ANONYMOUS LOGON") OR
(EventCode=4624 Logon_Type=9 LogonProcessName=seclogo)
| stats count min(_time) as firstTime max(_time) as lastTime by EventCode, Logon_Type, WorkstationName, user, dest
| `security_content_ctime(firstTime)`| `security_content_ctime(lastTime)`
| `detect_activity_related_to_pass_the_hash_attacks_filter` '
suppress:
suppress_fields: dest
suppress_period: 86400s
eli5: To detect pass the hash activity, we look at all events with event code 4624
that specify a logon type 3 (network logons) for remote pass the hash attacks and
logon type 9 for local pass the hash attacks. The search also filters out
events with an account name of 'Anonymous' to help reduce false positives.
entities:
- dest
how_to_implement: To successfully implement this search, you must ingest your Windows
Security Event logs and leverage the latest TA for Windows.
id: f5939373-8054-40ad-8c64-cec478a22a4b
investigations:
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd76
name: Get Authentication Logs For Endpoint
type: splunk
- id: fdcfb369-1725-4c24-824a-22972d7f0d55
name: Get Risk Modifiers For User
type: splunk
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd71
name: Get Process Info
type: splunk
- id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
name: Get Notable History
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
name: Get Notable Info
type: splunk
- id: fdcfb369-1725-4c24-824a-22972d7f0d65
name: Get Risk Modifiers For Endpoint
type: splunk
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd74
name: Get User Information from Identity Table
type: splunk
known_false_positives: Legitimate logon activity by authorized NTLM systems may be
detected by this search. Please investigate as appropriate.
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
- company: Splunk
email: pbareiss@splunk.com
name: Patrick Bareiss
mappings:
cis20:
- CIS 3
- CIS 5
- CIS 16
kill_chain_phases:
- Actions on Objectives
mitre_attack:
- Lateral Movement
- Pass the Hash
mitre_technique_id:
- T1075
nist:
- PR.PT
- PR.AT
- PR.AC
- PR.IP
modification_date: '2020-01-09'
name: Detect Activity Related to Pass the Hash Attacks
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
references: []
security_domain: access
spec_version: 2
type: splunk
version: '3.0'
detections/add_to_untrust_cert_store.yml
-115
View File
@@ -1,115 +0,0 @@
asset_type: Endpoint
confidence: high
creation_date: '2018-04-09'
data_metadata:
data_models:
- Endpoint
data_source:
- Endpoint Intel
providing_technologies:
- Carbon Black Response
- CrowdStrike Falcon
- Sysmon
- Tanium
- Ziften
description: Attempt to add a certificate to the untrusted certificate store
detect:
splunk:
correlation_rule:
notable:
nes_fields: dest, user, process_name
rule_description: Attempt to add a certificate to the untrusted certificate
store
rule_title: Attempt To Add Certificate to Untrusted Store
risk:
risk_object: dest
risk_object_type:
- system
risk_score: 50
schedule:
cron_schedule: 0 * * * *
earliest_time: -70m@m
latest_time: -10m@m
search: '| tstats `security_content_summariesonly` count min(_time) as firstTime values(Processes.process)
as process max(_time) as lastTime from datamodel=Endpoint.Processes where
Processes.process_name=certutil.exe (Processes.process=*-addstore* AND Processes.process=*disallowed*
) by Processes.parent_process Processes.process_name Processes.user | `drop_dm_object_name("Processes")`
| `security_content_ctime(firstTime)`|`security_content_ctime(lastTime)`'
suppress:
suppress_fields: process, dest
suppress_period: 86400s
eli5: Attackers will often attempt to disable security tools in order to evade detection.
It is also possible for end users to attempt to disable anti-virus or other security
tools to circumvent restrictions they encounter while trying to execute other programs.
One way malware may accomplish this is by adding the legitimate certificate used
to sign the security software to the untrusted certificate store. This will cause
the system to no longer trust the software signed with this certificate and disallow
it from executing. This search simply looks for the execution of **certutil.exe**
with the parameters `-addcert` and `disallowed`, which add a certification to the
"untrusted" certificate store.
entities:
- dest
how_to_implement: You must be ingesting data that records process activity from your
hosts to populate the Endpoint data model in the Processes node. You must also be
ingesting logs with both the process name and command line from your endpoints.
The command-line arguments are mapped to the "process" field in the Endpoint data
model.
id: 6bc5243e-ef36-45dc-9b12-f4a6be131159
investigations:
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd76
name: Get Authentication Logs For Endpoint
type: splunk
- id: fecf2918-670d-4f1c-872b-3d7317a41bf9
name: Get Parent Process Info
type: splunk
- id: fdcfb369-1725-4c24-824a-22972d7f0d55
name: Get Risk Modifiers For User
type: splunk
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd71
name: Get Process Info
type: splunk
- id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
name: Get Notable History
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
name: Get Notable Info
type: splunk
- id: fdcfb369-1725-4c24-824a-22972d7f0d65
name: Get Risk Modifiers For Endpoint
type: splunk
- id: bc91a8cf-35e7-4bb2-8140-e756cc06fd74
name: Get User Information from Identity Table
type: splunk
known_false_positives: There may be legitimate reasons for administrators to add a
certificate to the untrusted certificate store. In such cases, this will typically
be done on a large number of systems.
maintainers:
- company: Splunk
email: rvaldez@splunk.com
name: Rico Valdez
mappings:
cis20:
- CIS 3
- CIS 5
- CIS 8
kill_chain_phases:
- Installation
- Actions on Objectives
mitre_attack:
- Defense Evasion
- Disabling Security Tools
nist:
- PR.PT
- DE.CM
- PR.IP
modification_date: '2018-11-15'
name: Attempt To Add Certificate To Untrusted Store
original_authors:
- company: Splunk
email: rvaldez@splunk.com
name: Rico Valdez
references: []
security_domain: endpoint
spec_version: 2
type: splunk
version: '4.0'
detections/amazon_eks_kubernetes_cluster_scan_detection.yml
+27
View File
@@ -0,0 +1,27 @@
name: Amazon EKS Kubernetes cluster scan detection
id: 294c4686-63dd-4fe6-93a2-ca807626704a
version: 1
date: '2020-04-15'
description: This search provides information of unauthenticated requests via user
agent, and authentication data against Kubernetes cluster in AWS
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudWatch
EKS Logs inputs.
type: ESCU
references: []
author: Rod Soto, Splunk
search: '`aws_cloudwatchlogs_eks` "user.username"="system:anonymous" userAgent!="AWS
Security Scanner" | rename sourceIPs{} as src_ip | stats count min(_time) as firstTime
max(_time) as lastTime values(responseStatus.reason) values(source) as cluster_name
values(responseStatus.code) values(userAgent) as http_user_agent values(verb) values(requestURI)
by src_ip user.username user.groups{} | `security_content_ctime(lastTime)` | `security_content_ctime(firstTime)`
|`amazon_eks_kubernetes_cluster_scan_detection_filter` '
known_false_positives: Not all unauthenticated requests are malicious, but frequency,
UA and source IPs will provide context.
tags:
analytics_story:
- Kubernetes Scanning Activity
kill_chain_phases:
- Reconnaissance
security_domain: threat
asset_type: Amazon EKS Kubernetes cluster
detections/amazon_eks_kubernetes_pod_scan_detection.yml
+28
View File
@@ -0,0 +1,28 @@
name: Amazon EKS Kubernetes Pod scan detection
id: dbfca1dd-b8e5-4ba4-be0e-e565e5d62002
version: 1
date: '2020-04-15'
description: This search provides detection information on unauthenticated requests
against Kubernetes' Pods API
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on forAWS (version 4.4.0 or later), then configure your AWS CloudWatch
EKS Logs.Please also customize the `kubernetes_pods_aws_scan_fingerprint_detection`
macro to filter out the false positives.
type: ESCU
references: []
author: Rod Soto, Splunk
search: '`aws_cloudwatchlogs_eks` "user.username"="system:anonymous" verb=list objectRef.resource=pods
requestURI="/api/v1/pods" | rename source as cluster_name sourceIPs{} as src_ip
| stats count min(_time) as firstTime max(_time) as lastTime values(responseStatus.reason)
values(responseStatus.code) values(userAgent) values(verb) values(requestURI) by
src_ip cluster_name user.username user.groups{} | `security_content_ctime(lastTime)`
| `security_content_ctime(firstTime)` | `amazon_eks_kubernetes_pod_scan_detection_filter` '
known_false_positives: Not all unauthenticated requests are malicious, but frequency,
UA and source IPs and direct request to API provide context.
tags:
analytics_story:
- Kubernetes Scanning Activity
kill_chain_phases:
- Reconnaissance
security_domain: threat
asset_type: Amazon EKS Kubernetes cluster Pod
detections/anomalous_webclick.yml
-101
View File
@@ -1,101 +0,0 @@
asset_type: account
confidence: medium
creation_date: '2018-07-12'
data_metadata:
data_source:
- Network Communications
data_sourcetypes:
- stream:http
providing_technologies:
- Splunk Stream
description: This search is used to examine web sessions to identify those where the
clicks are occurring too quickly for a human or are occurring with a near-perfect
cadence (high periodicity or low standard deviation), resembling a script driven
session.
detect:
splunk:
correlation_rule:
notable:
nes_fields: session_id
rule_description: This search is used to examine web sessions in order to
identify unnaturally rapid clicks with near-perfect cadence (high periodicity
or low standard deviation), which resemble a script-driven session.
rule_title: 'Web Fraud Detection: Anomalous User Clickspeed'
risk:
risk_object: session_id
risk_object_type:
- other
risk_score: 40
schedule:
cron_schedule: 0 * * * *
earliest_time: -70m@m
latest_time: -10m@m
search: sourcetype=stream:http http_content_type=text* | rex field=cookie "form_key=(?<session_id>\w+)"
| streamstats window=2 current=1 range(_time) as TimeDelta by session_id |
where TimeDelta>0 |stats count stdev(TimeDelta) as ClickSpeedStdDev avg(TimeDelta)
as ClickSpeedAvg by session_id | where count>5 AND (ClickSpeedStdDev<.5 OR
ClickSpeedAvg<.5)
suppress:
suppress_fields: session_id
suppress_period: 3600s
eli5: It's suspicious when someone or something is moving throughout your website
too quickly or with a perfect click cadence. Fortunately, it's easy to detect by
calculating the time between clicks for each session and highlighting the anomalous
behavior.
entities:
- session_id
how_to_implement: Start with a dataset that allows you to see clickstream data for
each user click on the website. That data must have a time stamp and must contain
a reference to the session identifier being used by the website. This ties the clicks
together into clickstreams. This value is usually found in the http cookie. With
a bit of tuning, a version of this search could be used in high-volume scenarios,
such as scraping, crawling, application DDOS, credit-card testing, account takeover,
etc. Common data sources used for this detection are customized Apache logs, customized
IIS, and Splunk Stream.
id: 31337bbb-bc22-4752-b599-ef192df2dc7a
investigations:
- id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
name: Get Notable History
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
name: Get Notable Info
type: splunk
- id: bc91a8cf-35e7-4bb2-1120-e756cc06fd89
name: Get Web Session Information via session_id
type: splunk
- id: 5df39b3f-447d-4869-b673-8f45ad4616fe
name: Get Emails From Specific Sender
type: splunk
known_false_positives: As is common with many fraud-related searches, we are usually
looking to attribute risk or synthesize relevant context with loosly written detections
that simply detect anamoluous behavior.
maintainers:
- company: Splunk
email: Mayhem@splunk.com
name: Jim Apger
mappings:
cis20:
- CIS 6
kill_chain_phases:
- Actions on Objectives
mitre_attack:
- Initial Access
- Valid Accounts
nist:
- DE.AE
- DE.CM
modification_date: '2018-10-08'
name: Web Fraud - Anomalous User Clickspeed
original_authors:
- company: Splunk
email: Mayhem@splunk.com
name: Jim Apger
references:
- https://en.wikipedia.org/wiki/Session_ID
- https://en.wikipedia.org/wiki/Session_(computer_science)
- https://en.wikipedia.org/wiki/HTTP_cookie
- https://splunkbase.splunk.com/app/1809/
security_domain: threat
spec_version: 2
type: splunk
version: '1.0'
detections/api_acitivity_from_previously_unseen_user_role.yml
-97
View File
@@ -1,97 +0,0 @@
asset_type: AWS Instance
baselines:
- id: fc0edc95-fq2c-48b0-9f6f-63da3289fd03
name: Previously seen API call per user roles in CloudTrail
type: splunk
confidence: medium
creation_date: '2018-04-01'
data_metadata:
data_source:
- AWS CloudTrail logs
data_sourcetypes:
- aws:cloudtrail
providing_technologies:
- AWS
description: This search detects new API calls that have either never been seen before
or that have not been seen in the previous hour, where the identity type is `AssumedRole`.
detect:
splunk:
correlation_rule:
notable:
nes_fields: user
rule_description: A new API call made by $user$ has been detected. This API
activity has either never been seen before or has not been seen within the
last hour.
rule_title: New API call by $user$ detected
risk:
risk_object: user
risk_object_type:
- user
risk_score: 10
schedule:
cron_schedule: 30 * * * *
earliest_time: -70m@m
latest_time: -10m@m
search: sourcetype=aws:cloudtrail eventType=AwsApiCall errorCode=success userIdentity.type=AssumedRole
[search sourcetype=aws:cloudtrail eventType=AwsApiCall errorCode=success userIdentity.type=AssumedRole
| stats earliest(_time) as earliest latest(_time) as latest by userName eventName
| inputlookup append=t previously_seen_api_calls_from_user_roles | stats
min(earliest) as earliest, max(latest) as latest by userName eventName | outputlookup
previously_seen_api_calls_from_user_roles| eval newApiCallfromUserRole=if(earliest>=relative_time(now(),
"-70m@m"), 1, 0) | where newApiCallfromUserRole=1 | `security_content_ctime(earliest)` | `security_content_ctime(latest)`
| table eventName userName] |rename userName as user| stats values(eventName)
earliest(_time) as earliest latest(_time) as latest by user | `security_content_ctime(earliest)`
| `security_content_ctime(latest)`
suppress:
suppress_fields: user
suppress_period: 86400s
eli5: The subsearch will execute first and return the user roles and names of the
API calls completed within the last hour, where the type of user identity is `AssumedRole`.
It then appends the historical data to those results in the lookup file. Next, it
recalculates the `earliest` and `latest` fields for each user role, as well as the
name of the API call, and returns only those roles and API calls that have first
been seen in the past hour. This is combined with the main search to return the
values of API calls, name of the user role, and the earliest and latest time of
this activity. It is worth noting that the name of the role of a particular user
is parsed as "userName" in the CloudTrail logs.
entities:
- user
how_to_implement: You must install the AWS App for Splunk (version 5.1.0 or later)
and Splunk Add-on for AWS (version 4.4.0 or later), then configure your CloudTrail
inputs. This search works best when you run the "Previously seen API call per user
roles in CloudTrail" support search once to create a history of previously seen
user roles.
id: 22773e84-bac0-4595-b086-20d3f335b4f1
investigations:
- id: 3d6c3213-5fff-4a1e-b57d-b24c262171e7
name: Get Notable History
type: splunk
- id: f3fb4d1b-5f33-4b01-b541-c7af9534c242
name: Get Notable Info
type: splunk
- id: bc91a8cd-35e7-4bb2-6140-e756cc46fd76
name: Investigate AWS User Activities by user field
type: splunk
known_false_positives: It is possible that there are legitimate user roles making
new or infrequently used API calls in your infrastructure, causing the search to
trigger.
maintainers:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
mappings:
cis20:
- CIS 1
nist:
- ID.AM
modification_date: '2018-04-16'
name: Detect new API calls from user roles
original_authors:
- company: Splunk
email: bpatel@splunk.com
name: Bhavin Patel
references: []
security_domain: endpoint
spec_version: 2
type: splunk
version: '1.0'

Some files were not shown because too many files have changed in this diff Show More