Windows Events to ATT&CK Techniques¶

Author: Jose Rodriguez (@Cyb3rPandah)
Project: Infosec Jupyter Book
Public Organization: Open Threat Research
License: Creative Commons Attribution-ShareAlike 4.0 International
Reference: https://github.com/hunters-forge/OSSEM/tree/master/attack_data_sources

Importing Libraries¶

from attackcti import attack_client

import pandas as pd
from pandas import json_normalize
# Do not truncate Pandas output
pd.set_option('display.max_colwidth', None)

import requests

import yaml

Getting ATT&CK enterprise techniques for Windows platform¶

Getting all Windows techniques

lift = attack_client()
windowsTechniques = lift.get_techniques_by_platform('Windows',stix_format=False)
windowsTechniques = lift.remove_revoked(windowsTechniques)
windowsTechniques = json_normalize(windowsTechniques)
windowsTechniques = windowsTechniques[['tactic','technique_id','technique','data_sources']]
windowsTechniques.head()

	tactic	technique_id	technique	data_sources
0	[impact]	T1531	Account Access Removal	[Windows event logs, Process command-line parameters, Process monitoring]
1	[credential-access]	T1539	Steal Web Session Cookie	[File monitoring, API monitoring]
2	[impact]	T1529	System Shutdown/Reboot	[Windows event logs, Process command-line parameters, Process monitoring]
3	[discovery]	T1518	Software Discovery	[Process command-line parameters, Process monitoring, File monitoring]
4	[lateral-movement]	T1534	Internal Spearphishing	[SSL/TLS inspection, DNS records, Anti-virus, Web proxy, File monitoring, Mail server, Office 365 trace logs]

Splitting data_sources

windowsTechniques = windowsTechniques['data_sources'].apply(pd.Series)\
.merge(windowsTechniques, left_index = True, right_index = True)\
.drop(["data_sources"], axis = 1)\
.melt(id_vars = ['tactic','technique_id','technique'], value_name = "data_sources")\
.drop("variable", axis = 1)\
.dropna(subset=['data_sources'])
windowsTechniques.head()

	tactic	technique_id	technique	data_sources
0	[impact]	T1531	Account Access Removal	Windows event logs
1	[credential-access]	T1539	Steal Web Session Cookie	File monitoring
2	[impact]	T1529	System Shutdown/Reboot	Windows event logs
3	[discovery]	T1518	Software Discovery	Process command-line parameters
4	[lateral-movement]	T1534	Internal Spearphishing	SSL/TLS inspection

Getting OSSEM ATT&CK data sources modeling file¶

Getting Yaml File content

yamlUrl = 'https://raw.githubusercontent.com/hunters-forge/OSSEM/master/attack_data_sources/event-mappings/all_data_sources.yml'
dataSourcesModelingData = requests.get(yamlUrl)
all_ds = yaml.safe_load(dataSourcesModelingData.text)

Creating dictionary of data sources mapped to event IDs

all_data_sources = {}
# Create DS Keys
for ds_record in all_ds:
    ds_list = ds_record['data_source'].split(", ")
    for ds in ds_list:
        if ds not in all_data_sources.keys():
            all_data_sources[ds] = []
        if ds_record['event_id'] not in all_data_sources[ds]:
            all_data_sources[ds].append(ds_record['event_id'])

Generating dataframe

all_data_sources = pd.DataFrame(list(all_data_sources.items()), columns=['data_sources', 'event_ids'])
all_data_sources

	data_sources	event_ids
0	Authentication logs	[4776, 4771, 4624, 4648]
1	File monitoring	[6, 9, 11, 2, 5145, 4656, 4663, 4670, 4660, 4658]
2	DLL monitoring	[7]
3	Process use of network	[3, 5031, 5154, 5155, 5156, 5157, 5158, 5159]
4	Windows event logs	[4768, 4769, 4770, 4773, 4779, 4778, 4800, 4801, 4741, 4742, 4743, 4749, 4750, 4751, 4752, 4753, 4731, 4732, 4733, 4734, 4735, 4764, 4799, 4720, 4722, 4723, 4724, 4725, 4726, 4738, 4740, 4767, 4781, 4798, 4662, 4661, 5136, 5137, 5138, 5139, 5141, 4625, 5140, 5142, 5143, 5144, 4656, 4664, 4698, 4699, 4700, 4701, 4702, 4660, 4663, 4658, 4670, 4657, 4717, 4718, 4674, 4673, 5025, 5034, 4697, 4776, 4771, 4624, 4648]
5	Windows Registry	[4657, 12, 14, 13]
6	Process monitoring	[4688, 1, 4689, 5, 8, 10]
7	Process command-line parameters	[4688, 1]
8	Loaded DLLs	[7]
9	Named Pipes	[17, 18]
10	DNS records	[22]

Mapping Techniques to Event Logs¶

Joining Windows ATT&CK Techniques & OSSEM

mapping = pd.merge(windowsTechniques, all_data_sources, on = 'data_sources', how = 'left')
mapping.head()

	tactic	technique_id	technique	data_sources	event_ids
0	[impact]	T1531	Account Access Removal	Windows event logs	[4768, 4769, 4770, 4773, 4779, 4778, 4800, 4801, 4741, 4742, 4743, 4749, 4750, 4751, 4752, 4753, 4731, 4732, 4733, 4734, 4735, 4764, 4799, 4720, 4722, 4723, 4724, 4725, 4726, 4738, 4740, 4767, 4781, 4798, 4662, 4661, 5136, 5137, 5138, 5139, 5141, 4625, 5140, 5142, 5143, 5144, 4656, 4664, 4698, 4699, 4700, 4701, 4702, 4660, 4663, 4658, 4670, 4657, 4717, 4718, 4674, 4673, 5025, 5034, 4697, 4776, 4771, 4624, 4648]
1	[credential-access]	T1539	Steal Web Session Cookie	File monitoring	[6, 9, 11, 2, 5145, 4656, 4663, 4670, 4660, 4658]
2	[impact]	T1529	System Shutdown/Reboot	Windows event logs	[4768, 4769, 4770, 4773, 4779, 4778, 4800, 4801, 4741, 4742, 4743, 4749, 4750, 4751, 4752, 4753, 4731, 4732, 4733, 4734, 4735, 4764, 4799, 4720, 4722, 4723, 4724, 4725, 4726, 4738, 4740, 4767, 4781, 4798, 4662, 4661, 5136, 5137, 5138, 5139, 5141, 4625, 5140, 5142, 5143, 5144, 4656, 4664, 4698, 4699, 4700, 4701, 4702, 4660, 4663, 4658, 4670, 4657, 4717, 4718, 4674, 4673, 5025, 5034, 4697, 4776, 4771, 4624, 4648]
3	[discovery]	T1518	Software Discovery	Process command-line parameters	[4688, 1]
4	[lateral-movement]	T1534	Internal Spearphishing	SSL/TLS inspection	NaN

Techniques –> Event IDs¶

T1112 Modify Registry

T1112 = mapping[mapping['technique_id'] == 'T1112']
T1112

	tactic	technique_id	technique	data_sources	event_ids
110	[defense-evasion]	T1112	Modify Registry	Windows Registry	[4657, 12, 14, 13]
324	[defense-evasion]	T1112	Modify Registry	File monitoring	[6, 9, 11, 2, 5145, 4656, 4663, 4670, 4660, 4658]
510	[defense-evasion]	T1112	Modify Registry	Process monitoring	[4688, 1, 4689, 5, 8, 10]
646	[defense-evasion]	T1112	Modify Registry	Process command-line parameters	[4688, 1]
721	[defense-evasion]	T1112	Modify Registry	Windows event logs	[4768, 4769, 4770, 4773, 4779, 4778, 4800, 4801, 4741, 4742, 4743, 4749, 4750, 4751, 4752, 4753, 4731, 4732, 4733, 4734, 4735, 4764, 4799, 4720, 4722, 4723, 4724, 4725, 4726, 4738, 4740, 4767, 4781, 4798, 4662, 4661, 5136, 5137, 5138, 5139, 5141, 4625, 5140, 5142, 5143, 5144, 4656, 4664, 4698, 4699, 4700, 4701, 4702, 4660, 4663, 4658, 4670, 4657, 4717, 4718, 4674, 4673, 5025, 5034, 4697, 4776, 4771, 4624, 4648]

Tactics –> Event IDs¶

Lateral Movement

lateral_movement = mapping[['data_sources','event_ids']][mapping['tactic'].apply(lambda x: 'lateral-movement' in x)]
lateral_movement = lateral_movement.drop_duplicates(subset='data_sources').reset_index(drop=True)
lateral_movement

	data_sources	event_ids
0	SSL/TLS inspection	NaN
1	Windows Error Reporting	NaN
2	PowerShell logs	NaN
3	File monitoring	[6, 9, 11, 2, 5145, 4656, 4663, 4670, 4660, 4658]
4	Authentication logs	[4776, 4771, 4624, 4648]
5	Process use of network	[3, 5031, 5154, 5155, 5156, 5157, 5158, 5159]
6	DNS records	[22]
7	Process monitoring	[4688, 1, 4689, 5, 8, 10]
8	API monitoring	NaN
9	Packet capture	NaN
10	Data loss prevention	NaN
11	Netflow/Enclave netflow	NaN
12	Third-party application logs	NaN
13	Anti-virus	NaN
14	Windows Registry	[4657, 12, 14, 13]
15	Web proxy	NaN
16	DLL monitoring	[7]
17	Process command-line parameters	[4688, 1]
18	Network protocol analysis	NaN
19	Mail server	NaN
20	Binary file metadata	NaN
21	Office 365 trace logs	NaN
22	Windows event logs	[4768, 4769, 4770, 4773, 4779, 4778, 4800, 4801, 4741, 4742, 4743, 4749, 4750, 4751, 4752, 4753, 4731, 4732, 4733, 4734, 4735, 4764, 4799, 4720, 4722, 4723, 4724, 4725, 4726, 4738, 4740, 4767, 4781, 4798, 4662, 4661, 5136, 5137, 5138, 5139, 5141, 4625, 5140, 5142, 5143, 5144, 4656, 4664, 4698, 4699, 4700, 4701, 4702, 4660, 4663, 4658, 4670, 4657, 4717, 4718, 4674, 4673, 5025, 5034, 4697, 4776, 4771, 4624, 4648]

An opportunity to improve ATT&CK data sources mapping!!¶

As we can see in the example above (Lateral Movement techniques), the data source that brings more event logs is Windows event logs. However, this data source has a broad scope. We can split this data source in more detailed new data sources.

lateral_movement[lateral_movement['data_sources'] == 'Windows event logs']

	data_sources	event_ids
22	Windows event logs	[4768, 4769, 4770, 4773, 4779, 4778, 4800, 4801, 4741, 4742, 4743, 4749, 4750, 4751, 4752, 4753, 4731, 4732, 4733, 4734, 4735, 4764, 4799, 4720, 4722, 4723, 4724, 4725, 4726, 4738, 4740, 4767, 4781, 4798, 4662, 4661, 5136, 5137, 5138, 5139, 5141, 4625, 5140, 5142, 5143, 5144, 4656, 4664, 4698, 4699, 4700, 4701, 4702, 4660, 4663, 4658, 4670, 4657, 4717, 4718, 4674, 4673, 5025, 5034, 4697, 4776, 4771, 4624, 4648]

The OSSEM Project