Branch was auto-updated.

This commit is contained in:
github-actions[bot]
2021-02-23 14:18:13 +00:00
committed by GitHub
14 changed files with 745 additions and 610 deletions
+2 -2
View File
@@ -80,7 +80,7 @@ jobs:
command: |
cd security-content
source venv/bin/activate
python bin/validate.py --path . --verbose
python contentctl.py --path . --verbose validate
- run:
name: run doc-gen
command: |
@@ -118,7 +118,7 @@ jobs:
command: |
cd security-content
source venv/bin/activate
python bin/generate.py --path . --output package -v
python contentctl.py --path . --verbose generate --output package
# make a copy of use_case_lib in order to have ES work :-(
cp package/default/use_case_library.conf package/default/analyticstories.conf
- run:
+6
View File
@@ -1,3 +1,9 @@
# Ignore example files from contentctl tool
detections/*/.yml.example
stories/*.yml.example
tests/*/*.yml.example
# usual mac files
.DS_Store
#vim files
+43 -25
View File
@@ -15,7 +15,7 @@
<a href="https://github.com/splunk/security_content/stargazers">
<img src="https://img.shields.io/github/stars/splunk/security_content?style=social" /></a>
</p>
# Splunk Security Content
![security_content](docs/static/logo.png)
=====
@@ -24,11 +24,14 @@ Welcome to the Splunk Security Content
This project gives you access to our repository of Analytic Stories that are security guides the provide background on TTPs, mapped to the MITRE framework, the Lockheed Martin Kill Chain, and CIS controls. They include Splunk searches, machine-learning algorithms, and Splunk Phantom playbooks (where available)—all designed to work together to detect, investigate, and respond to threats.
# Usage🛡
The Splunk Security Content can be used via:
# Get Content🛡
The latest Splunk Security Content can be obtained via:
#### [Splunk App](https://github.com/splunk/security_content/releases)
Grab the latest release of DA-ESS-ContentUpdate and install it on a Splunk Enterprise instance. Alternatively, you can download it from [splunkbase](https://splunkbase.splunk.com/app/3449/), it is currently a Splunk Supported App.
#### [SSE App](https://splunkbase.splunk.com/app/3435/)
Grab the latest release of Splunk Security Essentials App and install it on a Splunk instance. You can download it from [splunkbase](https://splunkbase.splunk.com/app/3435/), it is a Splunk Supported App. SSE Splunk app today supports push updates for security content release, this is the **preferred way** to get content!
#### [ESCU App](https://github.com/splunk/security_content/releases)
Grab the latest release of DA-ESS-ContentUpdate.spl and install it on a Splunk instance. Alternatively, you can download it from [splunkbase](https://splunkbase.splunk.com/app/3449/), it is currently a Splunk Supported App.
#### [API](https://docs.splunkresearch.com/?version=latest)
```
@@ -38,35 +41,54 @@ curl -s https://content.splunkresearch.com | jq
}
```
#### [GitHub Workflow](https://github.com/splunk/security_content/wiki/Installation-and-Usage)
Create your customized version of Security Content by forking this project and following this [guide](https://github.com/splunk/security_content/wiki/Installation-and-Usage#github-workflow).
# Usage 🧰
### contentctl.py
The Content Control tool allows you to manipulate Splunk Security Content via the following actions:
# MITRE ATT&CK
1. **new** - Creates new content (detection, story, baseline)
2. **validate** - Validates written content
3. **generate** - Generates a deployment package for different platforms (splunk_app)
### pre-requisites
```
git clone git@github.com:splunk/security_content.git
cd security_content
pip install virtualenv
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
```
### create a new detection
`python contentctl.py new`
for a more indepth write up on how to write content see our [guide](https://github.com/splunk/security_content/wiki/Developing-Content).
### create a new analytic story
`python contentctl.py new -t story`
### validate written content a new analytic story
`python contentctl.py --verbose validate`
### generate a splunk app from current content
`python contentctl.py --path . --verbose generate --output package`
# MITRE ATT&CK ⚔️
### Detection Coverage
To view an up-to-date detection coverage map for all the content tagged with MITRE techniques visit: [https://mitremap.splunkresearch.com/](https://mitremap.splunkresearch.com/) under the **Detection Coverage** layer. Below is a snapshot in time of what technique we currently have some detection coverage for. The darker the shade of blue the more detections we have for this particular technique. This map is automatically updated on every release and generated from the [generate-coverage-map.py](https://github.com/splunk/security_content/blob/develop/bin/generate-coverage-map.py).
![](docs/mitre-map/coverage.png)
### Detection Priority by Threat Actors
If curious about how the Threat Research team prioritizes what content to build refer to our **Detection Priority by Threat Actors** layer. Using the actor data from [MITRE CTI](https://github.com/mitre/cti) we add a point for every threat actor that uses a particular technique, and then subtract a point of every detection we have mapped to that technique. The resulting map below is how we prioritize what techniques and detections to focus on next. This map is automatically updated on every release and is generated by the [generate-actors-map.py](https://github.com/splunk/security_content/blob/develop/bin/generate-actors-map.py) script.
If curious about how the Threat Research team prioritizes what content to build refer to our **Detection Priority by Threat Actors** layer in [https://mitremap.splunkresearch.com/](https://mitremap.splunkresearch.com/). Using the actor data from [MITRE CTI](https://github.com/mitre/cti) we add a point for every threat actor that uses a particular technique, and then subtract a point of every detection we have mapped to that technique. The resulting map below is how we prioritize what techniques and detections to focus on next. This map is automatically updated on every release and is generated by the [generate-actors-map.py](https://github.com/splunk/security_content/blob/develop/bin/generate-actors-map.py) script.
![](docs/mitre-map/priority.png)
# Customize to your Environment 🏗
Customize your content to change how [often detections run](https://github.com/splunk/security_content/wiki/Customize-to-Your-Environment#customizing-scheduling-and-alert-actions-with-deployments), or what the right source type for [sysmon](https://github.com/splunk/security_content/wiki/Customize-to-Your-Environment#customizing-source-types-with-macros) in your environment is please follow this [guide](https://github.com/splunk/security_content/wiki/Customize-to-Your-Environment).
# Writing Content 📓
Please see the Developing Content [guide](https://github.com/splunk/security_content/wiki/Developing-Content) for instructions.
# What's in an Analytic Story?
A complete use case, specifically built to detect, investigate, and respond to a specific threat like [Credential Dumping](https://github.com/splunk/security_content/blob/develop/stories/credential_dumping.yml) or [Ransomware](https://github.com/splunk/security_content/blob/develop/stories/ransomware.yml). A group of detections and a response make up an analytic story, they are associated with the tag `analytics_story: <name>`.
# Execute an Analytic Story 🏃‍♀️
Download and install the latest version of [Splunk Analytic Story Execution](https://github.com/splunk/analytic_story_execution/releases). This Splunk application will help the user do the following:
1. Execute an analytic story in an ad-hoc mode and view the results.
2. Schedule all the detection searches in an analytic story.
3. Update security_content via an API
# What's in an Analytic Story? 🗺
A complete use case, specifically built to detect, investigate, and respond to a specific threat like [Credential Dumping](https://github.com/splunk/security_content/blob/develop/stories/credential_dumping.yml) or [Ransomware](https://github.com/splunk/security_content/blob/develop/stories/ransomware.yml). A group of detections and a response make up an analytic story, they are associated with the tag `analytic_story: <name>`.
# Content Parts 🧩
@@ -80,10 +102,6 @@ Download and install the latest version of [Splunk Analytic Story Execution](htt
* [macros/](macros/): Implements Splunks search macros, shortcuts to commonly used search patterns like sysmon source type. More on how macros are used to customize content below.
* [lookups/](lookups/): Implements Splunks lookup, usually to provide a list of static values like commonly used ransomware extensions.
#### Supporting Parts
* [package/](package/): Splunk content app-source files, including lookups, binaries, and default config files
* [bin/](bin/): All binaries required to produce and test content
# Contribution 🥰
We welcome feedback and contributions from the community! Please see our [contributing to the project](https://github.com/splunk/security_content/wiki/Contributing-to-the-Project) for more information on how to get involved.
+3 -13
View File
@@ -602,21 +602,11 @@ def generate_mitre_lookup(OUTPUT_PATH):
def main(args):
def new(security_content_path, output, VERBOSE):
parser = argparse.ArgumentParser(description="generates splunk conf files out of security_content manifests", epilog="""
This tool converts manifests to the source files to be used by products like Splunk Enterprise.
It generates the savesearches.conf, analytics_stories.conf files for ES.""")
parser.add_argument("-p", "--path", required=True, help="path to security_content repo")
parser.add_argument("-o", "--output", required=True, help="path to the output directory")
parser.add_argument("-v", "--verbose", required=False, default=False, action='store_true', help="prints verbose output")
# parse them
args = parser.parse_args()
REPO_PATH = args.path
OUTPUT_PATH = args.output
REPO_PATH = security_content_path
OUTPUT_PATH = output
TEMPLATE_PATH = path.join(REPO_PATH, 'bin/jinja2_templates')
VERBOSE = args.verbose
stories = load_objects("stories/*.yml", VERBOSE, REPO_PATH)
macros = load_objects("macros/*.yml", VERBOSE, REPO_PATH)
lookups = load_objects("lookups/*.yml", VERBOSE, REPO_PATH)
+27
View File
@@ -0,0 +1,27 @@
name: {{name}}
id: {{uuid}}
version: 1
date: '{{date}}'
author: {{author}}
type: {{type}}
datamodel:
{% for datamodel in datamodels -%}
- {{datamodel}}
{% endfor -%}
description: {{description}}
search: '{{search}}'
how_to_implement: {{how_to_implement}}
known_false_positives: {{known_false_positives}}
references:
{% for reference in references -%}
- {{reference}}
{% endfor -%}
tags:
analytic_story:
- {{analytic_story_name}}
detections:
- {{detection_name}}
product:
{% for product in products -%}
- {{product}}
{% endfor -%}
+33
View File
@@ -0,0 +1,33 @@
name: {{name}}
id: {{uuid}}
version: 1
date: '{{date}}'
author: {{author}}
type: {{type}}
datamodel:
{% for datamodel in datamodels -%}
- {{datamodel}}
{% endfor -%}
description: {{description}}
search: '{{search}}'
how_to_implement: {{how_to_implement}}
known_false_positives: {{known_false_positives}}
references:
{% for reference in references -%}
- {{reference}}
{% endfor -%}
tags:
analytic_story:
- {{analytic_story_name}}
dataset:
- {{dataset_url}}
kill_chain_phases:
{% for kill_chain_phase in kill_chain_phases -%}
- {{kill_chain_phase}}
{% endfor -%}
mitre_attack_id:
- {{mitre_attack_id}}
product:
{% for product in products -%}
- {{product}}
{% endfor -%}
+24
View File
@@ -0,0 +1,24 @@
name: {{name}}
id: {{uuid}}
version: 1
date: '{{date}}'
author: {{author}}
type: {{type}}
description: {{description}}
narrative: {{narrative}}
references:
{% for reference in references -%}
- {{reference}}
{% endfor -%}
tags:
analytic_story:
- {{analytic_story_name}}
category:
{% for category in categories -%}
- {{category}}
{% endfor -%}
product:
{% for product in products -%}
- {{product}}
{% endfor -%}
usecase: {{usecase}}
+12
View File
@@ -0,0 +1,12 @@
name: {{name}}
tests:
- name: {{detection_name}}
file: {{detection_path}}
pass_condition: '{{pass_condition}}'
earliest_time: '{{earliest_time}}'
latest_time: '{{latest_time}}'
attack_data:
- file_name: {{file_name}}
data: {{dataset_url}}
source: {{splunk_source}}
sourcetype: {{splunk_sourcetype}}
+463
View File
@@ -0,0 +1,463 @@
#!/usr/bin/python
'''
Helps you create new Splunk Security Content.
'''
from pathlib import Path
from PyInquirer import prompt, Separator
import os
import getpass
from jinja2 import Environment, FileSystemLoader
import uuid
from datetime import date
from os import path
import sys
def detection_wizard(security_content_path,type,TEMPLATE_PATH):
questions = [
{
# get provider
'type': 'list',
'message': 'what kind of detection is this',
'name': 'detection_kind',
'choices': [
{
'name': 'endpoint'
},
{
'name': 'cloud'
},
{
'name': 'application'
},
{
'name': 'network'
},
{
'name': 'web'
},
{
'name': 'experimental'
},
],
'default': 'endpoint'
},
{
'type': 'input',
'message': 'enter detection name',
'name': 'detection_name',
'default': 'Powershell Encoded Command',
},
{
'type': 'input',
'message': 'enter author name',
'name': 'detection_author',
},
{
# get provider
'type': 'list',
'message': 'select a detection type',
'name': 'detection_type',
'choices': [
{
'name': 'batch'
},
{
'name': 'streaming'
},
],
'default': 'batch'
},
{
# get provider
'type': 'checkbox',
'message': 'select the datamodels used in the detection',
'name': 'datamodels',
'choices': [
{
'name': 'Endpoint',
'checked': True
},
{
'name': 'Network_Traffic'
},
{
'name': 'Authentication'
},
{
'name': 'Change'
},
{
'name': 'Change_Analysis'
},
{
'name': 'Email'
},
{
'name': 'Network_Resolution'
},
{
'name': 'Network_Traffic'
},
{
'name': 'Network_Sessions'
},
{
'name': 'Updates'
},
{
'name': 'Vulnerabilities'
},
{
'name': 'Web'
},
],
},
{
# get api_key
'type': 'input',
'message': 'enter search (spl)',
'name': 'detection_search',
'default': '| UPDATE_SPL'
},
{
# get api_key
'type': 'input',
'message': 'enter MITRE ATT&CK Technique IDs related to the detection, comma delimited for multiple',
'name': 'mitre_attack_ids',
'default': 'T1003.002'
},
{
# get provider
'type': 'checkbox',
'message': 'select kill chain phases related to the detection',
'name': 'kill_chain_phases',
'choices': [
{
'name': 'Reconnaissance'
},
{
'name': 'Intrusion'
},
{
'name': 'Exploitation',
'checked': True
},
{
'name': 'Privilege Escalation'
},
{
'name': 'Lateral Movement'
},
{
'name': 'Obfuscation'
},
{
'name': 'Denial of Service'
},
{
'name': 'Exfiltration'
},
],
},
]
answers = prompt(questions)
mitre_attack_id = answers['mitre_attack_ids'].split(',')
j2_env = Environment(loader=FileSystemLoader(TEMPLATE_PATH),
trim_blocks=True)
if answers['detection_type'] == 'batch':
answers['products'] = ['Splunk Enterprise','Splunk Enterprise Security','Splunk Cloud']
elif answers['detection_type'] == 'streaming':
answers['products'] = ['UEBA for Security Cloud']
# grab some vars for the test
detection_kind = answers['detection_kind']
# write a detection example
template = j2_env.get_template('detection.j2')
detection_name = answers['detection_name']
detection_file_name = detection_name.replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
output_path = path.join(security_content_path, 'detections/' + detection_kind + '/' + detection_file_name + '.yml')
output = template.render(uuid=uuid.uuid1(), date=date.today().strftime('%Y-%m-%d'),
author=answers['detection_author'], name=answers['detection_name'],
description='UPDATE_DESCRIPTION', how_to_implement='UPDATE_HOW_TO_IMPLEMENT', known_false_positives='UPDATE_KNOWN_FALSE_POSITIVES',
references='',datamodels=answers['datamodels'],
search= answers['detection_search'] + ' | `' + detection_file_name + '_filter`',
type=answers['detection_type'], analytic_story_name='UPDATE_STORY_NAME', mitre_attack_id = answers['mitre_attack_ids'],
kill_chain_phases=answers['kill_chain_phases'], dataset_url='UPDATE_DATASET_URL',
products=answers['products'])
with open(output_path, 'w', encoding="utf-8") as f:
f.write(output)
print("\n> contentctl wrote the detection to: {0}\n".format(output_path))
questions = [
{
'type': 'confirm',
'message': 'would you like to configure the test file for detection: {0}'.format(answers['detection_name']),
'name': 'continue',
'default': True,
},
{
'type': 'input',
'message': 'enter pass condition for the test of detection: {0}'.format(answers['detection_name']),
'name': 'pass_condition',
'default': '| stats count | where count > 0',
'when': lambda answers: answers['continue'],
},
{
'type': 'input',
'message': 'enter earliest_time for the test of detection: {0}'.format(answers['detection_name']),
'name': 'earliest_time',
'default': '-24h',
'when': lambda answers: answers['continue'],
},
{
'type': 'input',
'message': 'enter latest_time for the test of detection: {0}'.format(answers['detection_name']),
'name': 'latest_time',
'default': 'now',
'when': lambda answers: answers['continue'],
},
]
answers = prompt(questions)
if answers['continue']:
# and a corresponding test files
template = j2_env.get_template('test.j2')
test_name = detection_file_name + '.test.yml'
output_path = path.join(security_content_path, 'tests/' + detection_kind + '/' + test_name)
output = template.render(name=detection_name + ' Unit Test',
detection_name=detection_name,
detection_path='detections/' + detection_kind + '/' + detection_file_name + '.yml', pass_condition=answers['pass_condition'],
earliest_time=answers['earliest_time'], latest_time=answers['latest_time'], file_name='UPDATE_FILE_NAME',
splunk_source='UPDATE_SPLUNK_SOURCE',splunk_sourcetype='UPDATE_SPLUNK_SOURCETYPE',dataset_url='UPDATE_DATASET_URL')
with open(output_path, 'w', encoding="utf-8") as f:
f.write(output)
else:
# and a corresponding test files
template = j2_env.get_template('test.j2')
test_name = detection_file_name + '.test.yml'
output_path = path.join(security_content_path, 'tests/' + detection_kind + '/' + test_name)
output = template.render(name=detection_name + ' Unit Test',
detection_name=detection_name,
detection_path='detections/' + detection_kind + '/' + detection_file_name + '.yml', pass_condition='| stats count | where count > 0',
earliest_time='-24h', latest_time='now',file_name='UPDATE_FILE_NAME', splunk_source='UPDATE_SPLUNK_SOURCE',
splunk_sourcetype='UPDATE_SPLUNK_SOURCETYPE', dataset_url='UPDATE_DATASET_URL' )
with open(output_path, 'w', encoding="utf-8") as f:
f.write(output)
print("\n> contentctl wrote the test for this detection to: {0}\n".format(output_path))
def story_wizard(security_content_path,type, TEMPLATE_PATH):
questions = [
{
'type': 'input',
'message': 'enter story name',
'name': 'story_name',
'default': 'Suspicious Powershell Behavior',
},
{
'type': 'input',
'message': 'enter author name',
'name': 'story_author',
},
{
'type': 'list',
'message': 'select a story type',
'name': 'story_type',
'choices': [
{
'name': 'batch'
},
{
'name': 'streaming'
},
],
'default': 'batch'
},
{
'type': 'checkbox',
'message': 'select a category',
'name': 'category',
'choices': [
{
'name': 'Adversary Tactics',
'checked': True
},
{
'name': 'Account Compromise'
},
{
'name': 'Unauthorized Software'
},
{
'name': 'Best Practices'
},
{
'name': 'Cloud Security'
},
{
'name': 'Command and Control'
},
{
'name': 'Lateral Movement'
},
{
'name': 'Ransomware'
},
{
'name': 'Privilege Escalation'
},
],
},
{
# get provider
'type': 'list',
'message': 'select a use case',
'name': 'usecase',
'choices': [
{
'name': 'Advanced Threat Detection',
'checked': True
},
{
'name': 'Security Monitoring'
},
{
'name': 'Compliance'
},
{
'name': 'Insider Threat'
},
{
'name': 'Application Security'
},
{
'name': 'Other'
},
],
},
]
answers = prompt(questions)
j2_env = Environment(loader=FileSystemLoader(TEMPLATE_PATH),
trim_blocks=True)
if answers['story_type'] == 'batch':
answers['products'] = ['Splunk Enterprise','Splunk Enterprise Security','Splunk Cloud']
elif answers['story_type'] == 'streaming':
answers['products'] = ['UEBA for Security Cloud']
template = j2_env.get_template('story.j2')
story_name = answers['story_name']
story_file_name = story_name.replace(' ', '_').replace('-','_').replace('.','_').replace('/','_').lower()
output_path = path.join(security_content_path, 'stories/' + story_file_name + '.yml')
output = template.render(uuid=uuid.uuid1(), date=date.today().strftime('%Y-%m-%d'),
author=answers['story_author'], name=answers['story_name'], description='UPDATE_DESCRIPTION',
narrative='UPDATE_NARRATIVE', references=['https://www.destroyallsoftware.com/talks/wat'],
type=answers['story_type'], analytic_story_name=answers['story_name'],
categories=answers['category'], usecase=answers['usecase'], products=answers['products'])
with open(output_path, 'w', encoding="utf-8") as f:
f.write(output)
print("contentctl wrote a example story to: {0}".format(output_path))
def create_example(security_content_path,type, TEMPLATE_PATH):
getpass.getuser()
j2_env = Environment(loader=FileSystemLoader(TEMPLATE_PATH),
trim_blocks=True)
if type == 'detection':
# write a detection example
template = j2_env.get_template('detection.j2')
detection_name = getpass.getuser() + '_' + type + '.yml.example'
output_path = path.join(security_content_path, 'detections/endpoint/' + detection_name)
output = template.render(uuid=uuid.uuid1(), date=date.today().strftime('%Y-%m-%d'),
author='UPDATE_AUTHOR', name=getpass.getuser().capitalize() + ' ' + type.capitalize(),
description='UPDATE_DESCRIPTION',
how_to_implement='UPDATE_HOW_TO_IMPLENT',
known_false_positives='UPDATE_KNOWN_FALSE_POSITIVES',
references=['https://html5zombo.com/'],
datamodels=['Endpoint'], search='| UPDATE_SPL | `' + getpass.getuser() + '_' + type + '_filter`',
type='batch', analytic_story_name=' UPDATE_STORY_NAME', mitre_attack_id = 'T1003.01',
kill_chain_phases=['Exploitation'], dataset_url='UPDATE_DATASET_URL',
products=['Splunk Enterprise','Splunk Enterprise Security','Splunk Cloud'])
with open(output_path, 'w', encoding="utf-8") as f:
f.write(output)
print("contentctl wrote a example detection to: {0}".format(output_path))
# and a corresponding test files
template = j2_env.get_template('test.j2')
test_name = getpass.getuser() + '_' + type + '.test.yml.example'
output_path = path.join(security_content_path, 'tests/endpoint/' + test_name)
output = template.render(name=getpass.getuser().capitalize() + ' ' + type.capitalize() + ' Unit Test',
detection_name=getpass.getuser().capitalize() + ' ' + type.capitalize(),
detection_path='detections/endpoint/' + detection_name, pass_condition='| stats count | where count > 0',
earliest_time='-24h', latest_time='now', file_name='UPDATE_FILE_NAME', splunk_source='UPDATE_SPLUNK_SOURCE',
splunk_sourcetype='UPDATE_SPLUNK_SOURCETYPE',dataset_url='UPDATE_DATASET_URL')
with open(output_path, 'w', encoding="utf-8") as f:
f.write(output)
print("contentctl wrote a example test for this detection to: {0}".format(output_path))
elif type == 'story':
# write a story example
template = j2_env.get_template('story.j2')
story_name = getpass.getuser() + '_' + type + '.yml.example'
output_path = path.join(security_content_path, 'stories/' + story_name)
output = template.render(uuid=uuid.uuid1(), date=date.today().strftime('%Y-%m-%d'),
author='UPDATE_AUTHOR', name=getpass.getuser().capitalize() + ' ' + type.capitalize(),
description='UPDATE_DESCRIPTION',
narrative='UPDATE_NARRATIVE',
references=['https://www.destroyallsoftware.com/talks/wat'],
type='batch', analytic_story_name=getpass.getuser().capitalize() + ' ' + type.capitalize(),
categories=['Adversary Tactics'], usecase='Advanced Threat Detection', products=['Splunk Enterprise','Splunk Enterprise Security','Splunk Cloud'])
with open(output_path, 'w', encoding="utf-8") as f:
f.write(output)
print("contentctl wrote a example story to: {0}".format(output_path))
elif type == 'baseline':
# write a baseline example
template = j2_env.get_template('baseline.j2')
baseline_name = getpass.getuser() + '_' + type + '.yml.example'
output_path = path.join(security_content_path, 'baselines/' + baseline_name)
output = template.render(uuid=uuid.uuid1(), date=date.today().strftime('%Y-%m-%d'),
author='UPDATE_AUTHOR', name=getpass.getuser().capitalize() + ' ' + type.capitalize(),
description='UPDATE_DESCRIPTION',
how_to_implement='UPDATE_HOW_TO_IMPLENT',
known_false_positives='UPDATE_KNOWN_FALSE_POSITIVES',
references=['https://html5zombo.com/'],
datamodels=['Endpoint'], search='| UPDATE_SPL',
type='batch', analytic_story_name='UPDATE_STORY_NAME',
detection_name = 'UPDATE_DETECTION_NAME', dataset_url='UPDATE_DATASET_URL',
products=['Splunk Enterprise','Splunk Enterprise Security','Splunk Cloud'])
with open(output_path, 'w', encoding="utf-8") as f:
f.write(output)
print("contentctl wrote a example baseline to: {0}".format(output_path))
def new(security_content_path, VERBOSE, type, example_only):
valid_content_objects = ['detection','story', 'baseline']
if type not in valid_content_objects:
print("ERROR: content type: {0} is not valid, please use: {1}".format(type, str(valid_content_objects)))
sys.exit(1)
TEMPLATE_PATH = path.join(security_content_path, 'bin/jinja2_templates')
if example_only:
create_example(security_content_path,type, TEMPLATE_PATH)
sys.exit(0)
if type == 'detection':
detection_wizard(security_content_path, type, TEMPLATE_PATH)
elif type == 'story':
story_wizard(security_content_path, type, TEMPLATE_PATH)
print("WARNING do not forget to replace the UPDATE_* values with the correct information on the files!\ncompleted..")
+4 -14
View File
@@ -16,7 +16,7 @@ import re
from os import path, walk
def validate_schema(REPO_PATH, type, objects):
def validate_schema(REPO_PATH, type, objects, verbose):
error = False
errors = []
@@ -222,18 +222,8 @@ def validate_lookups_content(REPO_PATH, lookup_path, lookup):
return errors
if __name__ == "__main__":
# grab arguments
parser = argparse.ArgumentParser(description="validates security content manifest files", epilog="""
Validates security manifest for correctness, adhering to spec and other common items.
VALIDATE DOES NOT PROCESS RESPONSES SPEC for the moment.""")
parser.add_argument("-p", "--path", required=True, help="path to security-security content repo")
parser.add_argument("-v", "--verbose", required=False, action='store_true', help="prints verbose output")
# parse them
args = parser.parse_args()
REPO_PATH = args.path
verbose = args.verbose
def new(security_content_path, verbose):
REPO_PATH = security_content_path
validation_objects = ['macros','lookups','stories','detections','baselines','response_tasks','responses','deployments']
@@ -242,7 +232,7 @@ if __name__ == "__main__":
schema_errors = []
for validation_object in validation_objects:
objects, error, errors = validate_schema(REPO_PATH, validation_object, objects)
objects, error, errors = validate_schema(REPO_PATH, validation_object, objects, verbose)
schema_error = schema_error or error
if len(errors) > 0:
schema_errors = schema_errors + errors
+122
View File
@@ -0,0 +1,122 @@
import os
import sys
import argparse
from bin import validate as validator
from bin import generate as generator
from bin import newcontent as content
from pathlib import Path
VERSION = 1
def init(args):
path = args.path
print("""
Running Splunk Security Content Control Tool (contentctl) v{0}
starting program loaded for TIE Fighter...
_ _
T T T T
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | ____ | |
| | ___.r-"`--'"-r.____ | |
| |.-._,.,---~"_/_/ .----. \_\_"~---,.,_,-.| |
| ]|.[_]_ T~T[_.-Y / \ / \ Y-._]T~T _[_].|| |
[|-+[ ___]| [__ |-=[--()--]=-| __] |[___ ]+-|]
| ]|"[_] l_j[_"-l \ / \ / !-"_]l_j [_]~|| |
| |`-' "~"---.,_\\"\ "o--o" /"/_,.---"~" `-'| |
| | ~~"^-.____.-^"~~ | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
l_i l_j -Row
""".format(VERSION))
# parse config
security_content_path = Path(path).resolve()
if security_content_path.is_dir():
print("contentctl is reading from path {0}".format(
security_content_path))
else:
print("ERROR: contentctl failed to find security_content project")
sys.exit(1)
return str(security_content_path)
def new(args):
security_content_path = init(args)
print("contentctl is creating a new {0}".format(args.type))
content.new(security_content_path, args.verbose, args.type, args.example_only)
def validate(args):
security_content_path = init(args)
print("contentctl is validating all content under {0}".format(security_content_path))
validator.new(security_content_path, args.verbose)
def generate(args):
security_content_path = init(args)
output = Path(args.output).resolve()
if output.is_dir():
print("contentctl is using folder {0} to write deployment".format(
output))
else:
print("ERROR: contentctl failed to find folder for deployment {0}".format(output))
sys.exit(1)
print("contentctl is generating a new splunk_app under ".format(output))
generator.new(security_content_path, args.output, args.verbose)
def main(args):
# grab arguments
parser = argparse.ArgumentParser(
description="Use `contentctl.py action -h` to get help with any Splunk Security Content action")
parser.add_argument("-p", "--path", required=False, default=".",
help="path to the Splunk Security Content. Defaults to `.`")
parser.add_argument("--Version", default=False, action="version", version="version: {0}".format(VERSION),
help="shows current contentctl version")
parser.add_argument("-v", "--verbose", required=False, action='store_true',
help="silences all verbose output, defaults to False")
parser.set_defaults(func=lambda _: parser.print_help())
actions_parser = parser.add_subparsers(title="Splunk Security Content actions", dest="action")
new_parser = actions_parser.add_parser("new", help="Create new content (detection, story, baseline)")
validate_parser = actions_parser.add_parser("validate", help="Validates written content")
generate_parser = actions_parser.add_parser("generate", help="Generates a deployment package for different platforms (splunk_app)")
# new arguments
new_parser.add_argument("-t", "--type", required=False, type=str, default="detection",
help="Type of new content to create, please choose between `detection`, `baseline` or `story`. Defaults to `detection`")
new_parser.add_argument("-x", "--example_only", required=False, action='store_true',
help="Generates an example content UPDATE on the fields that need updating. Use `git status` to see what specific files are added. Skips new content wizard prompts.")
new_parser.set_defaults(func=new)
# validate arguments
validate_parser.set_defaults(func=validate, epilog="""
Validates security manifest for correctness, adhering to spec and other common items.
VALIDATE DOES NOT PROCESS RESPONSES SPEC for the moment.""")
# generate arguments
generate_parser.add_argument("-f", "--format", required=False, type=str, default="splunk_app",
help="Format of our deployment package, defaults to `splunk_app`.\n The deployment `splunk_app` runs on product Splunk Enterprise Security and Splunk Enterprise.")
generate_parser.add_argument("-o", "--output", required=False, type=str, default="package",
help="Path where to store the deployment package, defaults to `package`")
generate_parser.set_defaults(func=generate)
# # parse them
args = parser.parse_args()
return args.func(args)
if __name__ == "__main__":
main(sys.argv[1:])
-166
View File
@@ -1,166 +0,0 @@
{
"$id": "http://example.com/example.json",
"$schema": "http://json-schema.org/draft-07/schema",
"additionalProperties": true,
"description": "schema for detections",
"properties": {
"author": {
"$id": "#/properties/author",
"default": "",
"description": "Author of the detection",
"examples": [
"Patrick Bareiss, Splunk"
],
"type": "string"
},
"date": {
"$id": "#/properties/date",
"default": "",
"description": "date of creation or modification, format yyyy-mm-dd",
"examples": [
"2019-12-06"
],
"type": "string"
},
"description": {
"$id": "#/properties/description",
"default": "",
"description": "A detailed description of the detection",
"examples": [
"dbgcore.dll is a specifc DLL for Windows core debugging. It is used to obtain a memory dump of a process. This search detects the usage of this DLL for creating a memory dump of LSASS process. Memory dumps of the LSASS process can be created with tools such as Windows Task Manager or procdump."
],
"type": "string"
},
"how_to_implement": {
"$id": "#/properties/how_to_implement",
"default": "",
"description": "information about how to implement. Only needed for non standard implementations.",
"examples": [
"This search requires Sysmon Logs and a Sysmon configuration, which includes EventCode 10 for lsass.exe."
],
"type": "string"
},
"id": {
"$id": "#/properties/id",
"default": "",
"description": "UUID as unique identifier",
"examples": [
"fb4c31b0-13e8-4155-8aa5-24de4b8d6717"
],
"type": "string"
},
"known_false_positives": {
"$id": "#/properties/knwon_false_positives",
"default": "",
"description": "known false postives",
"examples": [
"Administrators can create memory dumps for debugging purposes, but memory dumps of the LSASS process would be unusual."
],
"type": "string"
},
"name": {
"$id": "#/properties/name",
"default": "",
"examples": [
"Access LSASS Memory for Dump Creation"
],
"title": "Name of detection",
"type": "string"
},
"references": {
"$id": "#/properties/references",
"additionalItems": true,
"default": [],
"description": "A list of references for this detection",
"examples": [
[
"https://2017.zeronights.org/wp-content/uploads/materials/ZN17_Kheirkhabarov_Hunting_for_Credentials_Dumping_in_Windows_Environment.pdf"
]
],
"items": {
"$id": "#/properties/references/items",
"default": "",
"description": "An explanation about the purpose of this instance.",
"examples": [
"https://2017.zeronights.org/wp-content/uploads/materials/ZN17_Kheirkhabarov_Hunting_for_Credentials_Dumping_in_Windows_Environment.pdf"
],
"title": "The Items Schema",
"type": "string"
},
"type": "array"
},
"search": {
"$id": "#/properties/search",
"default": "",
"description": "The Splunk search for the detection",
"examples": [
"`sysmon` EventCode=10 TargetImage=*lsass.exe CallTrace=*dbgcore.dll* OR CallTrace=*dbghelp.dll* | stats count min(_time) as firstTime max(_time) as lastTime by Computer, TargetImage, TargetProcessId, SourceImage, SourceProcessId | rename Computer as dest | `security_content_ctime(firstTime)`| `security_content_ctime(lastTime)` | `access_lsass_memory_for_dump_creation_filter`"
],
"type": "string"
},
"tags": {
"$id": "#/properties/tags",
"additionalProperties": true,
"default": {},
"description": "An array of key value pairs for tagging",
"examples": [
{
"analytics_story": "credential_dumping",
"kill_chain_phases": "Action on Objectives",
"mitre_attack_id": "T1078.004",
"cis20": "CIS 13",
"nist": "DE.DP",
"security domain": "network",
"asset_type": "AWS Instance",
"risk_object": "user",
"risk_object_type": "network_artifacts",
"risk score": "60",
"custom_key": "custom_value"
}
],
"minItems": 1,
"type": "object",
"uniqueItems": true
},
"type": {
"$id": "#/properties/type",
"default": "",
"description": "type of detection",
"examples": [
"ESCU"
],
"items": {
"enum": [
"ESCU",
"SSE",
"RBA"
],
"type": "string"
},
"type": "string"
},
"version": {
"$id": "#/properties/version",
"default": 0,
"description": "version of detection, e.g. 1 or 2 ...",
"examples": [
2
],
"type": "integer"
}
},
"required": [
"name",
"id",
"version",
"date",
"description",
"type",
"author",
"search",
"known_false_positives",
"tags"
],
"title": "Detection Schema",
"type": "object"
}
-390
View File
@@ -1,390 +0,0 @@
# Detection Schema Schema
```
http://example.com/example.json
```
schema for detections
| Abstract | Extensible | Status | Identifiable | Custom Properties | Additional Properties | Defined In |
|----------|------------|--------|--------------|-------------------|-----------------------|------------|
| Can be instantiated | No | Experimental | No | Forbidden | Permitted | |
# Detection Schema Properties
| Property | Type | Required | Nullable | Default | Defined by |
|----------|------|----------|----------|---------|------------|
| [author](#author) | `string` | **Required** | No | `""` | Detection Schema (this schema) |
| [date](#date) | `string` | **Required** | No | `""` | Detection Schema (this schema) |
| [description](#description) | `string` | **Required** | No | `""` | Detection Schema (this schema) |
| [how_to_implement](#how_to_implement) | `string` | Optional | No | `""` | Detection Schema (this schema) |
| [id](#id) | `string` | **Required** | No | `""` | Detection Schema (this schema) |
| [known_false_positives](#known_false_positives) | `string` | **Required** | No | `""` | Detection Schema (this schema) |
| [name](#name) | `string` | **Required** | No | `""` | Detection Schema (this schema) |
| [references](#references) | `string[]` | Optional | No | `[]` | Detection Schema (this schema) |
| [search](#search) | `string` | **Required** | No | `""` | Detection Schema (this schema) |
| [tags](#tags) | `object` | **Required** | No | `{}` | Detection Schema (this schema) |
| [type](#type) | `string` | **Required** | No | `""` | Detection Schema (this schema) |
| [version](#version) | `integer` | **Required** | No | `0` | Detection Schema (this schema) |
| `*` | any | Additional | Yes | this schema *allows* additional properties |
## author
Author of the detection
`author`
* is **required**
* type: `string`
* default: `""`
* defined in this schema
### author Type
`string`
### author Example
```json
"Patrick Bareiss, Splunk"
```
## date
date of creation or modification, format yyyy-mm-dd
`date`
* is **required**
* type: `string`
* default: `""`
* defined in this schema
### date Type
`string`
### date Example
```json
"2019-12-06"
```
## description
A detailed description of the detection
`description`
* is **required**
* type: `string`
* default: `""`
* defined in this schema
### description Type
`string`
### description Example
```json
"dbgcore.dll is a specifc DLL for Windows core debugging. It is used to obtain a memory dump of a process. This search detects the usage of this DLL for creating a memory dump of LSASS process. Memory dumps of the LSASS process can be created with tools such as Windows Task Manager or procdump."
```
## how_to_implement
information about how to implement. Only needed for non standard implementations.
`how_to_implement`
* is optional
* type: `string`
* default: `""`
* defined in this schema
### how_to_implement Type
`string`
### how_to_implement Example
```json
"This search requires Sysmon Logs and a Sysmon configuration, which includes EventCode 10 for lsass.exe."
```
## id
UUID as unique identifier
`id`
* is **required**
* type: `string`
* default: `""`
* defined in this schema
### id Type
`string`
### id Example
```json
"fb4c31b0-13e8-4155-8aa5-24de4b8d6717"
```
## known_false_positives
known false postives
`known_false_positives`
* is **required**
* type: `string`
* default: `""`
* defined in this schema
### known_false_positives Type
`string`
### known_false_positives Example
```json
"Administrators can create memory dumps for debugging purposes, but memory dumps of the LSASS process would be unusual."
```
## name
### Name of detection
`name`
* is **required**
* type: `string`
* default: `""`
* defined in this schema
### name Type
`string`
### name Example
```json
"Access LSASS Memory for Dump Creation"
```
## references
A list of references for this detection
`references`
* is optional
* type: `string[]`
* default: `[]`
* defined in this schema
### references Type
Array type: `string[]`
All items must be of the type:
`string`
An explanation about the purpose of this instance.
### references Example
```json
[
"https://2017.zeronights.org/wp-content/uploads/materials/ZN17_Kheirkhabarov_Hunting_for_Credentials_Dumping_in_Windows_Environment.pdf"
]
```
## search
The Splunk search for the detection
`search`
* is **required**
* type: `string`
* default: `""`
* defined in this schema
### search Type
`string`
### search Example
```json
"`sysmon` EventCode=10 TargetImage=*lsass.exe CallTrace=*dbgcore.dll* OR CallTrace=*dbghelp.dll* | stats count min(_time) as firstTime max(_time) as lastTime by Computer, TargetImage, TargetProcessId, SourceImage, SourceProcessId | rename Computer as dest | `security_content_ctime(firstTime)`| `security_content_ctime(lastTime)` | `access_lsass_memory_for_dump_creation_filter`"
```
## tags
An array of key value pairs for tagging
`tags`
* is **required**
* type: `object`
* default: `{}`
* defined in this schema
### tags Type
`object` with following properties:
| Property | Type | Required |
|----------|------|----------|
### tags Example
```json
{
"analytics_story": "credential_dumping",
"kill_chain_phases": "Action on Objectives",
"mitre_attack_id": "T1078.004",
"cis20": "CIS 13",
"nist": "DE.DP",
"security domain": "network",
"asset_type": "AWS Instance",
"risk_object": "user",
"risk_object_type": "network_artifacts",
"risk score": "60",
"custom_key": "custom_value"
}
```
## type
type of detection
`type`
* is **required**
* type: `string`
* default: `""`
* defined in this schema
### type Type
`string`
### type Example
```json
"ESCU"
```
## version
version of detection, e.g. 1 or 2 ...
`version`
* is **required**
* type: `integer`
* default: `0`
* defined in this schema
### version Type
`integer`
### version Example
```json
2
```
+6
View File
@@ -24,10 +24,14 @@ more-itertools==8.6.0
nodeenv==1.5.0
pathlib2==2.3.5
pre-commit==2.9.3
prompt-toolkit==1.0.14
Pygments==2.8.0
PyInquirer==1.0.3
pyrsistent==0.17.3
python-dateutil==2.8.1
pytz==2021.1
PyYAML==5.4.1
regex==2020.11.13
requests==2.25.1
scandir==1.10.0
semantic-version==2.8.5
@@ -36,9 +40,11 @@ six==1.15.0
sly==0.4
smmap==3.0.5
stix2==2.1.0
stix2-patterns==1.2.1
taxii2-client==2.2.2
toml==0.10.2
typing==3.7.4.3
urllib3==1.26.3
virtualenv==20.4.2
wcwidth==0.2.5
zipp==3.4.0