From 6139b72dac1581b6296d137d4a006ac8ff9dfc21 Mon Sep 17 00:00:00 2001 From: Namratha Sreekanta Date: Tue, 2 May 2023 14:25:07 -0700 Subject: [PATCH 1/9] initial commit for dns data exfil --- ...tration_using_pretrained_model_in_dsdl.yml | 112 +++++ ...ation_using_pretrained_model_in_dsdl.ipynb | 457 ++++++++++++++++++ ...ration_using_pretrained_model_in_dsdl.yaml | 3 + 3 files changed, 572 insertions(+) create mode 100644 detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml create mode 100644 notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb create mode 100644 notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yaml diff --git a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml new file mode 100644 index 0000000000..82c7d2fb74 --- /dev/null +++ b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -0,0 +1,112 @@ +name: Detect DNS Data Exfiltration using pretrained model in DSDL +id: 92f65c3a-968c-11ed-a1eb-0242ac120002 +version: 1 +date: "2023-04-27" +status: experimental +author: Abhinav Mishra, Kumar Sharad and Namratha Sreekanta, Splunk +type: Anomaly +datamodel: +- Network_Resolution +description: The following analytic uses a pre trained deep learning model to + detect DNS data exfiltration. The model is trained on the data we collected and is + inferred on live data. This detection detects low throughput DNS Tunneling (data exfiltration) using features computed from past events between + the same src and domain. The search uses macros from URL ToolBox app to generate features used by the model. + The model is a deep learning model that accepts DNS request as input along with a few custom features to generate a + pred_is_exfiltration_proba score. The higher the pred_is_exfiltration_proba, the more likely the DNS request is data exfiltration. + The threshold for flagging a request as DNS exfiltration is set at 0.5. +search: '| tstats `security_content_summariesonly` count min(_time) as firstTime + max(_time) as lastTime from datamodel=Network_Resolution where + DNS.message_type=response AND DNS.record_type=TXT by DNS.src DNS.dest + DNS.answer DNS.record_type + | `drop_dm_object_name("DNS")` + | rename answer as text + | fields firstTime, lastTime, message_type,record_type,src,dest, text + | apply detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl + | rename predicted_is_unknown as is_suspicious_score + | where is_suspicious_score > 0.5 + | `security_content_ctime(firstTime)` + | `security_content_ctime(lastTime)` + | rename src as "Source IP", dest as "Destination IP", text as "DNS Answer", record_type as "DNS Record Type" + | table "Source IP","Destination IP","DNS Answer","DNS Record Type", firstTime, lastTime,is_suspicious_score + | `detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl_filter`' + +how_to_implement: 'Steps to deploy detect DNS data exfiltration model into Splunk App + DSDL. This detection depends on the Splunk app for Data Science and Deep + Learning which can be found here - https://splunkbase.splunk.com/app/4607/ + and the Network Resolution datamodel which can be found here - + https://splunkbase.splunk.com/app/1621/. The detection uses a pre-trained + deep learning model that needs to be deployed in DSDL app. Follow the steps + for deployment here - `https://github.com/splunk/security_content/wiki/How-to-deploy-pre-trained-Deep-Learning-models-for-ESCU`.\ + + * Download the `artifacts .tar.gz` file from the link - https://seal.splunkresearch.com/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz + Download the `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb` + Jupyter notebook from https://github.com/splunk/security_content/notebooks\ + + * Login to the Jupyter Lab assigned for + detect_dns_data_exfiltration_using_pretrained_model_in_dsdl container. + This container should be listed on Containers page for DSDL app.\ + + * Below steps need to be followed inside Jupyter lab \ + + * Upload the detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz file + into `app/model/data` path using the upload option in the jupyter notebook.\ + + * Untar the artifact detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz using `tar -xf app/model/data/detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl.tar.gz -C app/model/data`\ + + * Upload detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pynb into Jupyter lab notebooks folder using the upload option in Jupyter lab \ + + * Save the notebook using the save option in jupyter notebook. \ + + * Upload `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.json` into `notebooks/data` folder.' +known_false_positives: False positives may be present if DNS data exfiltration request look very similar to benign DNS requests. +references: +- https://attack.mitre.org/techniques/T1048/003/ +- https://unit42.paloaltonetworks.com/dns-tunneling-how-dns-can-be-abused-by-malicious-actors/ +- https://en.wikipedia.org/wiki/Data_exfiltration +tags: + analytic_story: + - DNS Hijacking + - Suspicious DNS Traffic + - Command and Control + asset_type: Endpoint + cis20: + - CIS 8 + - CIS 12 + - CIS 13 + confidence: 90 + context: + - Source:Endpoint + - Stage:Command And Control + impact: 50 + kill_chain_phases: + - Command & Control + message: A DNS data exfiltration request was sent by this host $src$ , kindly review. + mitre_attack_id: + - T1048.003 + nist: + - PR.DS + - PR.PT + - DE.AE + - DE.CM + observable: + - name: query + type: Other + role: + - Attacker + - name: src + type: Hostname + role: + - Victim + product: + - Splunk Enterprise + - Splunk Enterprise Security + - Splunk Cloud + required_fields: + - _time + - DNS.message_type + - DNS.record_type + - DNS.src + - DNS.dest + - DNS.answer + risk_score: 45 + security_domain: network diff --git a/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb new file mode 100644 index 0000000000..0ef5631d7c --- /dev/null +++ b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb @@ -0,0 +1,457 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Deep Learning Toolkit for Splunk - Barebone Notebook" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "This notebook contains a barebone example workflow how to work on custom containerized code that seamlessly interfaces with the Deep Learning Toolkit for Splunk." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note: By default every time you save this notebook the cells are exported into a python module which is then invoked by Splunk MLTK commands like | fit ... | apply ... | summary . Please read the Model Development Guide in the Deep Learning Toolkit app for more information." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stage 0 - import libraries\n", + "At stage 0 we define all imports necessary to run our subsequent code depending on various libraries." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "deletable": false, + "name": "mltkc_import" + }, + "outputs": [], + "source": [ + "# this definition exposes all python module imports that should be available in all subsequent commands\n", + "import json\n", + "import numpy as np\n", + "import pandas as pd\n", + "import pandas as pd\n", + "import numpy as np\n", + "import torch\n", + "from torch.utils.data import DataLoader,TensorDataset,Dataset\n", + "from torch.autograd import Variable as V\n", + "import time\n", + "from sklearn.metrics import classification_report, confusion_matrix,roc_curve, auc\n", + "from sklearn.model_selection import train_test_split\n", + "import string\n", + "import torch.nn as nn\n", + "import collections\n", + "from sklearn.preprocessing import LabelEncoder\n", + "from sklearn.preprocessing import Normalizer, MinMaxScaler\n", + "from sklearn.pipeline import Pipeline\n", + "\n", + "import torch.optim as optim\n", + "MODEL_DIRECTORY = \"/srv/app/model/data/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", + "print(\"numpy version: \" + np.__version__)\n", + "print(\"pandas version: \" + pd.__version__)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stage 1 - get a data sample from Splunk\n", + "In Splunk run a search to pipe a dataset into your notebook environment. Note: mode=stage is used in the | fit command to do this." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "| makeresults count=10
\n", + "| streamstats c as i
\n", + "| eval s = i%3
\n", + "| eval feature_{s}=0
\n", + "| foreach feature_* [eval <<FIELD>>=random()/pow(2,31)]
\n", + "| fit MLTKContainer mode=stage algo=barebone epochs=10 batch_size=1 s from feature_* into app:barebone_model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "After you run this search your data set sample is available as a csv inside the container to develop your model. The name is taken from the into keyword (\"barebone_model\" in the example above) or set to \"default\" if no into keyword is present. This step is intended to work with a subset of your data to create your custom model." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "name": "mltkc_stage" + }, + "outputs": [], + "source": [ + "# this cell is not executed from MLTK and should only be used for staging data into the notebook environment\n", + "def stage(name):\n", + " with open(\"data/\"+name+\".csv\", 'r') as f:\n", + " df = pd.read_csv(f)\n", + " with open(\"data/\"+name+\".json\", 'r') as f:\n", + " param = json.load(f)\n", + " return df, param" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", + "df, param = stage(\"barebone_model\")\n", + "print(df.describe())\n", + "print(param)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stage 2 - create and initialize a model" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "deletable": false, + "name": "mltkc_init" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "94\n" + ] + } + ], + "source": [ + "# initialize your model\n", + "# available inputs: data and parameters\n", + "# returns the model object which will be used as a reference to call fit, apply and summary subsequently\n", + "\n", + "class DNSExfiltration(nn.Module):\n", + " def __init__(self,input_size):\n", + " super().__init__()\n", + " self.layer_1 = nn.Linear(input_size, 256) \n", + " self.layer_2 = nn.Linear(256, 256)\n", + " self.layer_out = nn.Linear(256, 1) \n", + " \n", + " \n", + " self.relu = nn.ReLU()\n", + " self.sigmoid = nn.Sigmoid()\n", + " self.dropout = nn.Dropout(p=0.5)\n", + " \n", + "\n", + " def forward(self, input):\n", + " x = self.relu(self.layer_1(input))\n", + " x = self.dropout(x)\n", + " x = self.relu(self.layer_2(x))\n", + " x = self.dropout(x)\n", + " x = self.sigmoid(self.layer_out(x))\n", + " return x\n", + " \n", + "\n", + "keys = list(string.printable.strip())\n", + "print (len(list(string.printable.strip())))\n", + " \n", + "def init(df,param):\n", + " model = DNSExfiltration(102)\n", + " model.load_state_dict(torch.load(MODEL_DIRECTORY+'/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pt',map_location=torch.device('cpu')))\n", + " model = model.to(device)\n", + " model.eval()\n", + " return model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "scrolled": true + }, + "outputs": [], + "source": [ + "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", + "print(init(df,param))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stage 3 - fit the model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "name": "mltkc_fit" + }, + "outputs": [], + "source": [ + "# train your model\n", + "# returns a fit info json object and may modify the model object\n", + "def fit(model,df,param):\n", + " # model.fit()\n", + " info = {\"message\": \"model trained\"}\n", + " return info" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", + "print(fit(model,df,param))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stage 4 - apply the model" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "deletable": false, + "name": "mltkc_apply" + }, + "outputs": [], + "source": [ + "# apply your model\n", + "# returns the calculated results\n", + "text_rows=[]\n", + "\n", + "def index_chars(x):\n", + " request_chars = {}\n", + " for i in range(len(x)):\n", + " request_chars[keys.index(x[i])] = request_chars.get(keys.index(x[i]),0)+1\n", + " text_rows.append(request_chars)\n", + " \n", + "def sub_domain_count(x):\n", + " \n", + " \n", + "def predict(dataloader,model):\n", + " predict_label = []\n", + " for batch in (dataloader):\n", + " \n", + " #convert to 1d tensor\n", + " predictions = model(batch.to(device))\n", + " output = (predictions >= 0.5).float() \n", + " predict_label.extend(output)\n", + " predict_label = [x.cpu().detach().numpy().item() for x in predict_label]\n", + " return output,predict_label\n", + "\n", + "\n", + "\n", + "def apply(model,df,param):\n", + " df['request'].apply(lambda x: index_chars(x))\n", + " text = pd.DataFrame(text_rows,columns=list(range(0, 94)))\n", + " text.reset_index(drop=True, inplace=True)\n", + " df.reset_index(drop=True, inplace=True)\n", + " text.fillna(0,inplace =True)\n", + " df = pd.concat([text,df],axis=1)\n", + " df = torch.FloatTensor(df.values)\n", + " # request without tld\n", + " df ['request_without_tld'] = [e.replace(k, '') for e, k in zip(df['request'].astype('str'), df[''].astype('str'))]\n", + " # length of domain\n", + " df[\"len\"] = df[\"request\"].apply(len)\n", + " \n", + " # number of subdomains\n", + " df[\"subdomains_count\"] = df[\"request\"].apply(lambda x: len(str(df[\"request\"]).split('.')))\n", + " \n", + " #\n", + " \n", + " dataloader = DataLoader(df, shuffle=True, batch_size=256)\n", + " predict_is_exfiltration_proba, predict_is_exfiltration = predict(train_dataloader)\n", + " output = pd.DataFrame()\n", + " output['pred_is_dns_data_exfiltration_proba'] = predict_is_exfiltration_proba\n", + " output['pred_is_dns_data_exfiltration'] = predict_is_exfiltration\n", + " return output" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", + "print(apply(model,df,param))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stage 5 - save the model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "name": "mltkc_save" + }, + "outputs": [], + "source": [ + "# save model to name in expected convention \"_\"\n", + "def save(model,name):\n", + " with open(MODEL_DIRECTORY + name + \".json\", 'w') as file:\n", + " json.dump(model, file)\n", + " return model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stage 6 - load the model" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "deletable": false, + "name": "mltkc_load" + }, + "outputs": [], + "source": [ + "# load model from name in expected convention \"_\"\n", + "def load(name):\n", + " model = DNSExfiltration(102)\n", + " model.load_state_dict(torch.load(MODEL_DIRECTORY+'/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pt',map_location=torch.device('cpu')))\n", + " model = model.to(device)\n", + " model.eval()\n", + " return model" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Stage 7 - provide a summary of the model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "name": "mltkc_summary" + }, + "outputs": [], + "source": [ + "# return a model summary\n", + "def summary(model=None):\n", + " returns = {\"version\": {\"numpy\": np.__version__, \"pandas\": pd.__version__} }\n", + " return returns" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "After implementing your fit, apply, save and load you can train your model:
\n", + "| makeresults count=10
\n", + "| streamstats c as i
\n", + "| eval s = i%3
\n", + "| eval feature_{s}=0
\n", + "| foreach feature_* [eval <<FIELD>>=random()/pow(2,31)]
\n", + "| fit MLTKContainer algo=barebone s from feature_* into app:barebone_model
" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Or apply your model:
\n", + "| makeresults count=10
\n", + "| streamstats c as i
\n", + "| eval s = i%3
\n", + "| eval feature_{s}=0
\n", + "| foreach feature_* [eval <<FIELD>>=random()/pow(2,31)]
\n", + "| apply barebone_model as the_meaning_of_life" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## End of Stages\n", + "All subsequent cells are not tagged and can be used for further freeform code" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.13" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yaml b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yaml new file mode 100644 index 0000000000..04dbc86ea1 --- /dev/null +++ b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yaml @@ -0,0 +1,3 @@ +description: A Jupyter notebook for detecting DNS data exfiltration using a pre-trained model in DSDL +filename: detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb +name: detect_dns_data_exfiltration_using_pretrained_model_in_dsdl \ No newline at end of file From e4a676f7927f48e1a842bd6b0ec6f5782d7188f0 Mon Sep 17 00:00:00 2001 From: Namratha Sreekanta Date: Fri, 19 May 2023 13:24:00 -0700 Subject: [PATCH 2/9] changing spl and notebook for feature transformations --- ...tration_using_pretrained_model_in_dsdl.yml | 31 ++- ...ation_using_pretrained_model_in_dsdl.ipynb | 263 +++++++++++++----- 2 files changed, 214 insertions(+), 80 deletions(-) diff --git a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml index 82c7d2fb74..acb373402e 100644 --- a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml +++ b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -14,21 +14,22 @@ description: The following analytic uses a pre trained deep learning model to The model is a deep learning model that accepts DNS request as input along with a few custom features to generate a pred_is_exfiltration_proba score. The higher the pred_is_exfiltration_proba, the more likely the DNS request is data exfiltration. The threshold for flagging a request as DNS exfiltration is set at 0.5. -search: '| tstats `security_content_summariesonly` count min(_time) as firstTime - max(_time) as lastTime from datamodel=Network_Resolution where - DNS.message_type=response AND DNS.record_type=TXT by DNS.src DNS.dest - DNS.answer DNS.record_type - | `drop_dm_object_name("DNS")` - | rename answer as text - | fields firstTime, lastTime, message_type,record_type,src,dest, text - | apply detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl - | rename predicted_is_unknown as is_suspicious_score - | where is_suspicious_score > 0.5 - | `security_content_ctime(firstTime)` - | `security_content_ctime(lastTime)` - | rename src as "Source IP", dest as "Destination IP", text as "DNS Answer", record_type as "DNS Record Type" - | table "Source IP","Destination IP","DNS Answer","DNS Record Type", firstTime, lastTime,is_suspicious_score - | `detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl_filter`' +search: '| tstats `security_content_summariesonly` count from datamodel=Network_Resolution by DNS.src _time DNS.query +| `drop_dm_object_name("DNS")` +| sort - _time,src, query +| streamstats count as count by src query +| where count < 10 +| table src,query,count,_time +| apply dns_exfil11 +| table src,_time, query,count,pred_is_dns_data_exfiltration_proba,pred_is_dns_data_exfiltration +| where count == 1 +| rename pred_is_dns_data_exfiltration_proba as is_exfiltration_score +| rename pred_is_dns_data_exfiltration as is_exfiltration +| where is_exfiltration_score > 0.5 +| `security_content_ctime(_time)` +| rename src as "Source IP" _time as "time" +| table "Source IP","time","query","is_exfiltration_score","is_exfiltration" +| `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter`' how_to_implement: 'Steps to deploy detect DNS data exfiltration model into Splunk App DSDL. This detection depends on the Splunk app for Data Science and Deep diff --git a/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb index 0ef5631d7c..e0fe8c2432 100644 --- a/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb +++ b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb @@ -4,14 +4,16 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Deep Learning Toolkit for Splunk - Barebone Notebook" + "# Detect DNS Exfiltration using Deep Learning" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "This notebook contains a barebone example workflow how to work on custom containerized code that seamlessly interfaces with the Deep Learning Toolkit for Splunk." + "This notebook uses a pre-trained model to detect if a DNS request is a DNS exfiltration request. The pre-trained model is a deep learning model that takes in current DNS request along with previous 'x' number of requests between the same src and subdomain. The deep learning layers consume text input along with additional features such as entropy, length for the current request and aggregated features such as entropy_avg and size_avg in past few requests. \n", + "\n", + "The model outputs a probability score and the threshold for determining whether it is DNS exfiltration request or not is set at 0.5. Higher the `pred_is_dns_data_exfiltration_proba` probability it is more likely the DNS request is exfiltration. The notebook has apply() method implemented and fit() method unimplemented since it uses pre-trained model for making predictions. " ] }, { @@ -31,7 +33,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 2, "metadata": { "deletable": false, "name": "mltkc_import" @@ -56,16 +58,26 @@ "from sklearn.preprocessing import LabelEncoder\n", "from sklearn.preprocessing import Normalizer, MinMaxScaler\n", "from sklearn.pipeline import Pipeline\n", - "\n", + "from collections import Counter\n", "import torch.optim as optim\n", - "MODEL_DIRECTORY = \"/srv/app/model/data/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl\"" + "import math\n", + "MODEL_DIRECTORY = \"/srv/app/model/data/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl/\"" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "numpy version: 1.22.1\n", + "pandas version: 1.5.1\n" + ] + } + ], "source": [ "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", "print(\"numpy version: \" + np.__version__)\n", @@ -101,7 +113,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "metadata": { "deletable": false, "name": "mltkc_stage" @@ -119,12 +131,29 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + " is_exfiltration rank\n", + "count 10.000000 10.000000\n", + "mean 4.800000 4.800000\n", + "std 3.084009 3.084009\n", + "min 1.000000 1.000000\n", + "25% 2.000000 2.000000\n", + "50% 4.500000 4.500000\n", + "75% 7.750000 7.750000\n", + "max 9.000000 9.000000\n", + "{'options': {'params': {'mode': 'stage', 'algo': 'dns_exfil5'}, 'args': ['is_exfiltration', 'src', 'query', 'rank'], 'target_variable': ['is_exfiltration'], 'feature_variables': ['src', 'query', 'rank'], 'model_name': 'dns_exfil5', 'algo_name': 'MLTKContainer', 'mlspl_limits': {'handle_new_cat': 'default', 'max_distinct_cat_values': '100', 'max_distinct_cat_values_for_classifiers': '100', 'max_distinct_cat_values_for_scoring': '100', 'max_fit_time': '600', 'max_inputs': '100000', 'max_memory_usage_mb': '4000', 'max_model_size_mb': '30', 'max_score_time': '600', 'use_sampling': 'true'}, 'kfold_cv': None}, 'feature_variables': ['src', 'query', 'rank'], 'target_variables': ['is_exfiltration']}\n" + ] + } + ], "source": [ "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", - "df, param = stage(\"barebone_model\")\n", + "df, param = stage(\"dns_exfil5\")\n", "print(df.describe())\n", "print(param)" ] @@ -138,7 +167,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 6, "metadata": { "deletable": false, "name": "mltkc_init" @@ -156,7 +185,7 @@ "# initialize your model\n", "# available inputs: data and parameters\n", "# returns the model object which will be used as a reference to call fit, apply and summary subsequently\n", - "\n", + "device='cpu'\n", "class DNSExfiltration(nn.Module):\n", " def __init__(self,input_size):\n", " super().__init__()\n", @@ -183,20 +212,35 @@ "print (len(list(string.printable.strip())))\n", " \n", "def init(df,param):\n", - " model = DNSExfiltration(102)\n", - " model.load_state_dict(torch.load(MODEL_DIRECTORY+'/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pt',map_location=torch.device('cpu')))\n", - " model = model.to(device)\n", + " model = DNSExfiltration(99)\n", + " model.load_state_dict(torch.load(MODEL_DIRECTORY+'detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pt',map_location=torch.device('cpu')))\n", + " model = model.to('cpu')\n", " model.eval()\n", " return model" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": { "scrolled": true }, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "DNSExfiltration(\n", + " (layer_1): Linear(in_features=99, out_features=256, bias=True)\n", + " (layer_2): Linear(in_features=256, out_features=256, bias=True)\n", + " (layer_out): Linear(in_features=256, out_features=1, bias=True)\n", + " (relu): ReLU()\n", + " (sigmoid): Sigmoid()\n", + " (dropout): Dropout(p=0.5, inplace=False)\n", + ")\n" + ] + } + ], "source": [ "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", "print(init(df,param))" @@ -211,7 +255,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 8, "metadata": { "deletable": false, "name": "mltkc_fit" @@ -228,12 +272,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 9, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'message': 'model trained'}\n" + ] + } + ], "source": [ "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", - "print(fit(model,df,param))" + "print(fit(None,None,None))" ] }, { @@ -245,7 +297,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 33, "metadata": { "deletable": false, "name": "mltkc_apply" @@ -254,66 +306,120 @@ "source": [ "# apply your model\n", "# returns the calculated results\n", - "text_rows=[]\n", + "text_rows = []\n", + "size_avg = []\n", + "entropy_avg = []\n", "\n", - "def index_chars(x):\n", - " request_chars = {}\n", - " for i in range(len(x)):\n", - " request_chars[keys.index(x[i])] = request_chars.get(keys.index(x[i]),0)+1\n", - " text_rows.append(request_chars)\n", - " \n", - "def sub_domain_count(x):\n", - " \n", - " \n", + "# this method accepts a dataloader and makes predictions in batches\n", "def predict(dataloader,model):\n", + " predict_label_proba = []\n", " predict_label = []\n", " for batch in (dataloader):\n", " \n", " #convert to 1d tensor\n", - " predictions = model(batch.to(device))\n", - " output = (predictions >= 0.5).float() \n", + " predictions = model(batch.to('cpu'))\n", + " output = (predictions >= 0.5).int() \n", + " predict_label_proba.extend(predictions)\n", " predict_label.extend(output)\n", " predict_label = [x.cpu().detach().numpy().item() for x in predict_label]\n", - " return output,predict_label\n", + " predict_label_proba = [x.cpu().detach().numpy().item() for x in predict_label_proba]\n", + " return predict_label_proba,predict_label\n", + " \n", + "# this method accepts a DNS request and converts into indexes based on printable characters\n", + "def index_chars(x):\n", + " request_chars = {}\n", + " for i in range(len(x)):\n", + " request_chars[keys.index(x[i])] = request_chars.get(keys.index(x[i]), 0) + 1\n", + " text_rows.append(request_chars)\n", + "\n", + "# calculates entropy of a domain\n", + "def entropy(domain):\n", + " p, lns = Counter(domain), float(len(domain))\n", + " return -sum(count / lns * math.log(count / lns, 2) for count in p.values())\n", "\n", "\n", + "# removes the subdomain/domain from the request\n", + "def replace_tld(x):\n", + " if x is not None:\n", + " return str(x).rsplit('.', 2)[0]\n", + " else:\n", + " return x\n", + " \n", + "# get the subdomain/domain from the request\n", + "def get_tld(x):\n", + " without_tld = str(x).rsplit('.', 2)[0]\n", + " return str(x).replace(without_tld,'').lstrip(\".\")\n", "\n", - "def apply(model,df,param):\n", - " df['request'].apply(lambda x: index_chars(x))\n", - " text = pd.DataFrame(text_rows,columns=list(range(0, 94)))\n", + "# compute aggregated features for the same src and subdomain/domain on a window of 10 events\n", + "def get_aggregated_features(row,df):\n", + " src = row['src']\n", + " tld = row['tld']\n", + " prev_events = df[(df['src']==src) & (df['tld']==tld)]\n", + " \n", + " size_avg.append(prev_events['len'].mean())\n", + " entropy_avg.append(prev_events['entropy'].mean())\n", + "\n", + "# prepare input df by calculating features\n", + "def prepare_input_df(df):\n", + " keys = list(string.printable.strip())\n", + "\n", + " \n", + " \n", + " df['query'].apply(lambda x: index_chars(x))\n", + " text = pd.DataFrame(text_rows, columns=list(range(0, 94)))\n", " text.reset_index(drop=True, inplace=True)\n", " df.reset_index(drop=True, inplace=True)\n", - " text.fillna(0,inplace =True)\n", - " df = pd.concat([text,df],axis=1)\n", - " df = torch.FloatTensor(df.values)\n", + " text.fillna(0, inplace=True)\n", + " df = pd.concat([text, df], axis=1)\n", + "\n", + "\n", " # request without tld\n", - " df ['request_without_tld'] = [e.replace(k, '') for e, k in zip(df['request'].astype('str'), df[''].astype('str'))]\n", + " df['request_without_domain'] = df['query'].apply(lambda row: replace_tld(row))\n", + "\n", + " # request without tld\n", + " df['tld'] = df['query'].apply(lambda row: get_tld(row))\n", + " \n", " # length of domain\n", - " df[\"len\"] = df[\"request\"].apply(len)\n", + " df['len'] = df['request_without_domain'].apply(len)\n", " \n", " # number of subdomains\n", - " df[\"subdomains_count\"] = df[\"request\"].apply(lambda x: len(str(df[\"request\"]).split('.')))\n", + " df['subdomains_count'] = df['request_without_domain'].apply(lambda x: len(str(x).split('.')))\n", " \n", - " #\n", + " # entropy\n", + " df['entropy'] = df['request_without_domain'].apply(lambda x: entropy(x))\n", " \n", - " dataloader = DataLoader(df, shuffle=True, batch_size=256)\n", - " predict_is_exfiltration_proba, predict_is_exfiltration = predict(train_dataloader)\n", - " output = pd.DataFrame()\n", - " output['pred_is_dns_data_exfiltration_proba'] = predict_is_exfiltration_proba\n", - " output['pred_is_dns_data_exfiltration'] = predict_is_exfiltration\n", + " # take most-recent request\n", + " recent_df = df.loc[df['count'] == 1]\n", + "\n", + " # calculate feature by aggregating events\n", + "\n", + " recent_df.apply(lambda x: get_aggregated_features(x,df),axis=1)\n", + " recent_df['size_avg'] = size_avg\n", + " recent_df['entropy_avg'] = entropy_avg\n", + " return recent_df\n", + "\n", + " \n", + "# apply model on processed dataframe to predict exfiltration\n", + "def apply(model,df,param):\n", + " df.drop(['_time'], axis=1,inplace=True)\n", + " recent_df = prepare_input_df(df)\n", + " input_df = recent_df.drop(['src' ,'query','count','request_without_domain','tld'], axis=1)\n", + " recent_df.drop(['request_without_domain','tld','len','subdomains_count','entropy','size_avg','entropy_avg'], axis=1, inplace=True)\n", + " recent_df.drop(range(0, 94),axis=1,inplace=True)\n", + " input_tensor = torch.FloatTensor(input_df.values)\n", + " dataloader = DataLoader(input_tensor, shuffle=True, batch_size=256)\n", + " predict_is_exfiltration_proba, predict_is_exfiltration = predict(dataloader,model)\n", + " recent_df['pred_is_dns_data_exfiltration_proba'] = predict_is_exfiltration_proba\n", + " recent_df['pred_is_dns_data_exfiltration'] = predict_is_exfiltration\n", + " print(recent_df.columns)\n", + " print(df.columns)\n", + " text_rows.clear()\n", + " size_avg.clear()\n", + " entropy_avg.clear()\n", + " output = pd.merge(recent_df,df,on=['src','query','count'],how='right')\n", " return output" ] }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# THIS CELL IS NOT EXPORTED - free notebook cell for testing or development purposes\n", - "print(apply(model,df,param))" - ] - }, { "cell_type": "markdown", "metadata": {}, @@ -346,7 +452,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 12, "metadata": { "deletable": false, "name": "mltkc_load" @@ -355,13 +461,40 @@ "source": [ "# load model from name in expected convention \"_\"\n", "def load(name):\n", - " model = DNSExfiltration(102)\n", - " model.load_state_dict(torch.load(MODEL_DIRECTORY+'/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pt',map_location=torch.device('cpu')))\n", - " model = model.to(device)\n", + " model = DNSExfiltration(99)\n", + " model.load_state_dict(torch.load(MODEL_DIRECTORY+'detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pt',map_location=torch.device('cpu')))\n", + " model = model.to('cpu')\n", " model.eval()\n", " return model" ] }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "DNSExfiltration(\n", + " (layer_1): Linear(in_features=99, out_features=256, bias=True)\n", + " (layer_2): Linear(in_features=256, out_features=256, bias=True)\n", + " (layer_out): Linear(in_features=256, out_features=1, bias=True)\n", + " (relu): ReLU()\n", + " (sigmoid): Sigmoid()\n", + " (dropout): Dropout(p=0.5, inplace=False)\n", + ")" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "load(\"\")" + ] + }, { "cell_type": "markdown", "metadata": {}, From d828cd5d6b075ab78bf58ebaf52257bf8789eae7 Mon Sep 17 00:00:00 2001 From: Namratha Sreekanta Date: Fri, 19 May 2023 13:42:07 -0700 Subject: [PATCH 3/9] adding json file --- ...tration_using_pretrained_model_in_dsdl.yml | 154 +++++++++--------- ...ration_using_pretrained_model_in_dsdl.json | 1 + ...ration_using_pretrained_model_in_dsdl.yml} | 0 3 files changed, 77 insertions(+), 78 deletions(-) create mode 100644 notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.json rename notebooks/{detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yaml => detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml} (100%) diff --git a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml index acb373402e..1cf93770a2 100644 --- a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml +++ b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -1,113 +1,111 @@ +--- name: Detect DNS Data Exfiltration using pretrained model in DSDL id: 92f65c3a-968c-11ed-a1eb-0242ac120002 version: 1 -date: "2023-04-27" +date: 2023-04-27 status: experimental author: Abhinav Mishra, Kumar Sharad and Namratha Sreekanta, Splunk type: Anomaly datamodel: -- Network_Resolution + - Network_Resolution description: The following analytic uses a pre trained deep learning model to - detect DNS data exfiltration. The model is trained on the data we collected and is - inferred on live data. This detection detects low throughput DNS Tunneling (data exfiltration) using features computed from past events between - the same src and domain. The search uses macros from URL ToolBox app to generate features used by the model. - The model is a deep learning model that accepts DNS request as input along with a few custom features to generate a - pred_is_exfiltration_proba score. The higher the pred_is_exfiltration_proba, the more likely the DNS request is data exfiltration. - The threshold for flagging a request as DNS exfiltration is set at 0.5. -search: '| tstats `security_content_summariesonly` count from datamodel=Network_Resolution by DNS.src _time DNS.query -| `drop_dm_object_name("DNS")` -| sort - _time,src, query -| streamstats count as count by src query -| where count < 10 -| table src,query,count,_time -| apply dns_exfil11 -| table src,_time, query,count,pred_is_dns_data_exfiltration_proba,pred_is_dns_data_exfiltration -| where count == 1 -| rename pred_is_dns_data_exfiltration_proba as is_exfiltration_score -| rename pred_is_dns_data_exfiltration as is_exfiltration -| where is_exfiltration_score > 0.5 -| `security_content_ctime(_time)` -| rename src as "Source IP" _time as "time" -| table "Source IP","time","query","is_exfiltration_score","is_exfiltration" -| `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter`' + detect DNS data exfiltration. The model is trained on the data we collected + and is inferred on live data. This detection detects low throughput DNS + Tunneling (data exfiltration) using features computed from past events between + the same src and domain. The search uses macros from URL ToolBox app to + generate features used by the model. The model is a deep learning model that + accepts DNS request as input along with a few custom features to generate a + pred_is_exfiltration_proba score. The higher the pred_is_exfiltration_proba, + the more likely the DNS request is data exfiltration. The threshold for + flagging a request as DNS exfiltration is set at 0.5. +search: '| tstats `security_content_summariesonly` count from + datamodel=Network_Resolution by DNS.src _time DNS.query | + `drop_dm_object_name("DNS")` | sort - _time,src, query | streamstats count as + count by src query | where count < 10 | table src,query,count,_time | apply + detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter | table + src,_time, + query,count,pred_is_dns_data_exfiltration_proba,pred_is_dns_data_exfiltration + | where count == 1 | rename pred_is_dns_data_exfiltration_proba as + is_exfiltration_score | rename pred_is_dns_data_exfiltration as + is_exfiltration | where is_exfiltration_score > 0.5 | + `security_content_ctime(_time)`| rename src as "Source IP" _time as "time" | + table "Source IP","time","query","is_exfiltration_score","is_exfiltration" | + `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter`' +how_to_implement: >- + Steps to deploy detect DNS data exfiltration model into Splunk + App DSDL. This detection depends on the Splunk app for Data Science and Deep + Learning which can be found here - https://splunkbase.splunk.com/app/4607/ and + the Network Resolution datamodel which can be found here - + https://splunkbase.splunk.com/app/1621/. The detection uses a pre-trained deep + learning model that needs to be deployed in DSDL app. Follow the steps for + deployment here - + `https://github.com/splunk/security_content/wiki/How-to-deploy-pre-trained-Deep-Learning-models-for-ESCU`.\ -how_to_implement: 'Steps to deploy detect DNS data exfiltration model into Splunk App - DSDL. This detection depends on the Splunk app for Data Science and Deep - Learning which can be found here - https://splunkbase.splunk.com/app/4607/ - and the Network Resolution datamodel which can be found here - - https://splunkbase.splunk.com/app/1621/. The detection uses a pre-trained - deep learning model that needs to be deployed in DSDL app. Follow the steps - for deployment here - `https://github.com/splunk/security_content/wiki/How-to-deploy-pre-trained-Deep-Learning-models-for-ESCU`.\ + * Download the `artifacts .tar.gz` file from the link - https://seal.splunkresearch.com/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz Download the `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb` Jupyter notebook from https://github.com/splunk/security_content/notebooks\ - * Download the `artifacts .tar.gz` file from the link - https://seal.splunkresearch.com/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz - Download the `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb` - Jupyter notebook from https://github.com/splunk/security_content/notebooks\ - - * Login to the Jupyter Lab assigned for - detect_dns_data_exfiltration_using_pretrained_model_in_dsdl container. - This container should be listed on Containers page for DSDL app.\ + * Login to the Jupyter Lab assigned for detect_dns_data_exfiltration_using_pretrained_model_in_dsdl container. This container should be listed on Containers page for DSDL app.\ * Below steps need to be followed inside Jupyter lab \ - * Upload the detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz file - into `app/model/data` path using the upload option in the jupyter notebook.\ + * Upload the detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz file into `app/model/data` path using the upload option in the jupyter notebook.\ - * Untar the artifact detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz using `tar -xf app/model/data/detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl.tar.gz -C app/model/data`\ + * Untar the artifact detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.tar.gz using `tar -xf app/model/data/detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl.tar.gz -C app/model/data`\ - * Upload detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pynb into Jupyter lab notebooks folder using the upload option in Jupyter lab \ + * Upload detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pynb into Jupyter lab notebooks folder using the upload option in Jupyter lab \ - * Save the notebook using the save option in jupyter notebook. \ + * Save the notebook using the save option in jupyter notebook. \ - * Upload `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.json` into `notebooks/data` folder.' -known_false_positives: False positives may be present if DNS data exfiltration request look very similar to benign DNS requests. + * Upload `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.json` into `notebooks/data` folder. +known_false_positives: False positives may be present if DNS data exfiltration + request look very similar to benign DNS requests. references: -- https://attack.mitre.org/techniques/T1048/003/ -- https://unit42.paloaltonetworks.com/dns-tunneling-how-dns-can-be-abused-by-malicious-actors/ -- https://en.wikipedia.org/wiki/Data_exfiltration + - https://attack.mitre.org/techniques/T1048/003/ + - https://unit42.paloaltonetworks.com/dns-tunneling-how-dns-can-be-abused-by-malicious-actors/ + - https://en.wikipedia.org/wiki/Data_exfiltration tags: analytic_story: - - DNS Hijacking - - Suspicious DNS Traffic - - Command and Control + - DNS Hijacking + - Suspicious DNS Traffic + - Command and Control asset_type: Endpoint cis20: - - CIS 8 - - CIS 12 - - CIS 13 + - CIS 8 + - CIS 12 + - CIS 13 confidence: 90 context: - Source:Endpoint - Stage:Command And Control impact: 50 kill_chain_phases: - - Command & Control + - Command & Control message: A DNS data exfiltration request was sent by this host $src$ , kindly review. mitre_attack_id: - - T1048.003 + - T1048.003 nist: - - PR.DS - - PR.PT - - DE.AE - - DE.CM + - PR.DS + - PR.PT + - DE.AE + - DE.CM observable: - - name: query - type: Other - role: - - Attacker - - name: src - type: Hostname - role: - - Victim + - name: query + type: Other + role: + - Attacker + - name: src + type: Hostname + role: + - Victim product: - - Splunk Enterprise - - Splunk Enterprise Security - - Splunk Cloud + - Splunk Enterprise + - Splunk Enterprise Security + - Splunk Cloud required_fields: - - _time - - DNS.message_type - - DNS.record_type - - DNS.src - - DNS.dest - - DNS.answer + - _time + - DNS.message_type + - DNS.record_type + - DNS.src + - DNS.dest + - DNS.answer risk_score: 45 - security_domain: network + security_domain: network \ No newline at end of file diff --git a/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.json b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.json new file mode 100644 index 0000000000..78e38139a9 --- /dev/null +++ b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.json @@ -0,0 +1 @@ +{"options": {"params": {"mode": "stage", "algo": "detect_dns_data_exfiltration_using_pretrained_model_in_dsdl"}, "args": ["is_exfiltration", "src", "query", "rank"], "target_variable": ["is_exfiltration"], "feature_variables": ["src", "query", "rank"], "model_name": "detect_dns_data_exfiltration_using_pretrained_model_in_dsdl", "algo_name": "MLTKContainer", "mlspl_limits": {"handle_new_cat": "default", "max_distinct_cat_values": "100", "max_distinct_cat_values_for_classifiers": "100", "max_distinct_cat_values_for_scoring": "100", "max_fit_time": "600", "max_inputs": "100000", "max_memory_usage_mb": "4000", "max_model_size_mb": "30", "max_score_time": "600", "use_sampling": "true"}, "kfold_cv": null}, "feature_variables": ["src", "query", "rank"], "target_variables": ["is_exfiltration"]} \ No newline at end of file diff --git a/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yaml b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml similarity index 100% rename from notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yaml rename to notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml From 2f947af695e29fa90aaa6653859fcc13a1044771 Mon Sep 17 00:00:00 2001 From: patel-bhavin Date: Tue, 23 May 2023 14:07:22 -0700 Subject: [PATCH 4/9] fix up yaml for validation --- ...tration_using_pretrained_model_in_dsdl.yml | 24 ++++--------------- 1 file changed, 4 insertions(+), 20 deletions(-) diff --git a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml index 1cf93770a2..a8970bae3d 100644 --- a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml +++ b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -1,13 +1,11 @@ ---- name: Detect DNS Data Exfiltration using pretrained model in DSDL -id: 92f65c3a-968c-11ed-a1eb-0242ac120002 +id: 92f65c3a-968c-11ed-a1eb-0242ac120012 version: 1 -date: 2023-04-27 +date: '2023-04-27' status: experimental author: Abhinav Mishra, Kumar Sharad and Namratha Sreekanta, Splunk type: Anomaly -datamodel: - - Network_Resolution +data_source: [] description: The following analytic uses a pre trained deep learning model to detect DNS data exfiltration. The model is trained on the data we collected and is inferred on live data. This detection detects low throughput DNS @@ -31,7 +29,7 @@ search: '| tstats `security_content_summariesonly` count from `security_content_ctime(_time)`| rename src as "Source IP" _time as "time" | table "Source IP","time","query","is_exfiltration_score","is_exfiltration" | `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter`' -how_to_implement: >- +how_to_implement: Steps to deploy detect DNS data exfiltration model into Splunk App DSDL. This detection depends on the Splunk app for Data Science and Deep Learning which can be found here - https://splunkbase.splunk.com/app/4607/ and @@ -68,25 +66,11 @@ tags: - Suspicious DNS Traffic - Command and Control asset_type: Endpoint - cis20: - - CIS 8 - - CIS 12 - - CIS 13 confidence: 90 - context: - - Source:Endpoint - - Stage:Command And Control impact: 50 - kill_chain_phases: - - Command & Control message: A DNS data exfiltration request was sent by this host $src$ , kindly review. mitre_attack_id: - T1048.003 - nist: - - PR.DS - - PR.PT - - DE.AE - - DE.CM observable: - name: query type: Other From bc20c9c2c081a48c49f576b78ff877ddaf1962e1 Mon Sep 17 00:00:00 2001 From: Namratha Sreekanta Date: Thu, 8 Jun 2023 12:43:07 -0700 Subject: [PATCH 5/9] addressing review feedback --- ...s_data_exfiltration_using_pretrained_model_in_dsdl.yml | 8 ++++---- ...ous_dns_txt_records_using_pretrained_model_in_dsdl.yml | 5 +---- 2 files changed, 5 insertions(+), 8 deletions(-) diff --git a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml index a8970bae3d..cb2db08a07 100644 --- a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml +++ b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -19,15 +19,15 @@ description: The following analytic uses a pre trained deep learning model to search: '| tstats `security_content_summariesonly` count from datamodel=Network_Resolution by DNS.src _time DNS.query | `drop_dm_object_name("DNS")` | sort - _time,src, query | streamstats count as - count by src query | where count < 10 | table src,query,count,_time | apply + rank by src query | where rank < 10 | table src,query,rank,_time | apply detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter | table src,_time, query,count,pred_is_dns_data_exfiltration_proba,pred_is_dns_data_exfiltration - | where count == 1 | rename pred_is_dns_data_exfiltration_proba as + | where rank == 1 | rename pred_is_dns_data_exfiltration_proba as is_exfiltration_score | rename pred_is_dns_data_exfiltration as is_exfiltration | where is_exfiltration_score > 0.5 | - `security_content_ctime(_time)`| rename src as "Source IP" _time as "time" | - table "Source IP","time","query","is_exfiltration_score","is_exfiltration" | + `security_content_ctime(_time)`| table src, _time as "time" | + table src, _time,query,is_exfiltration_score,is_exfiltration | `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter`' how_to_implement: Steps to deploy detect DNS data exfiltration model into Splunk diff --git a/detections/network/detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl.yml b/detections/network/detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl.yml index 37a824f7c7..55f59b217c 100644 --- a/detections/network/detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl.yml +++ b/detections/network/detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl.yml @@ -22,10 +22,7 @@ search: '| tstats `security_content_summariesonly` count min(_time) as firstTime | rename answer as text | fields firstTime, lastTime, message_type,record_type,src,dest, text | apply detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl | rename predicted_is_unknown as is_suspicious_score | where is_suspicious_score > - 0.5 | `security_content_ctime(firstTime)` | `security_content_ctime(lastTime)` | - rename src as "Source IP", dest as "Destination IP", text as "DNS Answer", record_type - as "DNS Record Type" | table "Source IP","Destination IP","DNS Answer","DNS Record - Type", firstTime, lastTime,is_suspicious_score | `detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl_filter`' + 0.5 | `security_content_ctime(firstTime)` | `security_content_ctime(lastTime)` | table src,dest,text,record_type, firstTime, lastTime,is_suspicious_score | `detect_suspicious_dns_txt_records_using_pretrained_model_in_dsdl_filter`' how_to_implement: 'Steps to deploy detect suspicious DNS TXT records model into Splunk App DSDL. This detection depends on the Splunk app for Data Science and Deep Learning which can be found here - `https://splunkbase.splunk.com/app/4607/` and the Network From 05885bc6f2edc091dbc508063b612362dd9b61c2 Mon Sep 17 00:00:00 2001 From: Namratha Sreekanta Date: Thu, 8 Jun 2023 14:00:28 -0700 Subject: [PATCH 6/9] changing the search --- ...ns_data_exfiltration_using_pretrained_model_in_dsdl.yml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml index cb2db08a07..a0056b166f 100644 --- a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml +++ b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -20,13 +20,12 @@ search: '| tstats `security_content_summariesonly` count from datamodel=Network_Resolution by DNS.src _time DNS.query | `drop_dm_object_name("DNS")` | sort - _time,src, query | streamstats count as rank by src query | where rank < 10 | table src,query,rank,_time | apply - detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter | table - src,_time, - query,count,pred_is_dns_data_exfiltration_proba,pred_is_dns_data_exfiltration + detect_dns_data_exfiltration_using_pretrained_model_in_dsdl | table + src,_time,query,rank,pred_is_dns_data_exfiltration_proba,pred_is_dns_data_exfiltration | where rank == 1 | rename pred_is_dns_data_exfiltration_proba as is_exfiltration_score | rename pred_is_dns_data_exfiltration as is_exfiltration | where is_exfiltration_score > 0.5 | - `security_content_ctime(_time)`| table src, _time as "time" | + `security_content_ctime(_time)` | table src, _time,query,is_exfiltration_score,is_exfiltration | `detect_dns_data_exfiltration_using_pretrained_model_in_dsdl_filter`' how_to_implement: From 07829301373acb9de771677bb1e2521af3a4e28c Mon Sep 17 00:00:00 2001 From: Namratha Sreekanta Date: Mon, 12 Jun 2023 13:56:20 -0700 Subject: [PATCH 7/9] Adding missing lookup files --- ...ion_using_pretrained_model_in_dsdl.mlmodel | 2 ++ ...tration_using_pretrained_model_in_dsdl.yml | 4 ++++ ...ation_using_pretrained_model_in_dsdl.ipynb | 23 ++++++++----------- 3 files changed, 16 insertions(+), 13 deletions(-) create mode 100644 lookups/__mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.mlmodel create mode 100644 lookups/__mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml diff --git a/lookups/__mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.mlmodel b/lookups/__mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.mlmodel new file mode 100644 index 0000000000..4d61fec35b --- /dev/null +++ b/lookups/__mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.mlmodel @@ -0,0 +1,2 @@ +algo,model,options +MLTKContainer,"{""__mlspl_type"": [""mltkc.MLTKContainer"", ""MLTKContainer""], ""dict"": {""endpoint_url"": ""https://localhost:62645"", ""out_params"": {""params"": {""mode"": ""stage"", ""algo"": ""detect_dns_data_exfiltration_using_pretrained_model_in_dsdl""}, ""args"": [""is_exfiltration"", ""src"", ""query"", ""rank""], ""target_variable"": [""is_exfiltration""], ""feature_variables"": [""src"", ""query"", ""rank""], ""model_name"": ""detect_dns_data_exfiltration_using_pretrained_model_in_dsdl"", ""algo_name"": ""MLTKContainer"", ""mlspl_limits"": {""handle_new_cat"": ""default"", ""max_distinct_cat_values"": ""100"", ""max_distinct_cat_values_for_classifiers"": ""100"", ""max_distinct_cat_values_for_scoring"": ""100"", ""max_fit_time"": ""600"", ""max_inputs"": ""100000"", ""max_memory_usage_mb"": ""4000"", ""max_model_size_mb"": ""30"", ""max_score_time"": ""600"", ""use_sampling"": ""true""}, ""kfold_cv"": null}, ""feature_variables"": [""src"", ""query"", ""rank""], ""target_variable"": ""is_exfiltration""}}","{""params"": {""mode"": ""stage"", ""algo"": ""detect_dns_data_exfiltration_using_pretrained_model_in_dsdl""}, ""args"": [""is_exfiltration"", ""src"", ""query"", ""rank""], ""target_variable"": [""is_exfiltration""], ""feature_variables"": [""src"", ""query"", ""rank""], ""model_name"": ""detect_dns_data_exfiltration_using_pretrained_model_in_dsdl"", ""algo_name"": ""MLTKContainer"", ""mlspl_limits"": {""handle_new_cat"": ""default"", ""max_distinct_cat_values"": ""100"", ""max_distinct_cat_values_for_classifiers"": ""100"", ""max_distinct_cat_values_for_scoring"": ""100"", ""max_fit_time"": ""600"", ""max_inputs"": ""100000"", ""max_memory_usage_mb"": ""4000"", ""max_model_size_mb"": ""30"", ""max_score_time"": ""600"", ""use_sampling"": ""true""}, ""kfold_cv"": null}" diff --git a/lookups/__mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/lookups/__mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml new file mode 100644 index 0000000000..f834f44dc7 --- /dev/null +++ b/lookups/__mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -0,0 +1,4 @@ +description: Detect DNS Data Exfiltration using pretrained Model in DSDL +filename: __mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.mlmodel +name: __mlspl_detect_dns_data_exfiltration_using_pretrained_model_in_dsdl +case_sensitive_match: 'false' \ No newline at end of file diff --git a/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb index e0fe8c2432..81319defd8 100644 --- a/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb +++ b/notebooks/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.ipynb @@ -189,9 +189,9 @@ "class DNSExfiltration(nn.Module):\n", " def __init__(self,input_size):\n", " super().__init__()\n", - " self.layer_1 = nn.Linear(input_size, 256) \n", - " self.layer_2 = nn.Linear(256, 256)\n", - " self.layer_out = nn.Linear(256, 1) \n", + " self.layer_1 = nn.Linear(input_size, 128) \n", + " self.layer_2 = nn.Linear(128, 128)\n", + " self.layer_out = nn.Linear(128, 1) \n", " \n", " \n", " self.relu = nn.ReLU()\n", @@ -212,7 +212,7 @@ "print (len(list(string.printable.strip())))\n", " \n", "def init(df,param):\n", - " model = DNSExfiltration(99)\n", + " model = DNSExfiltration(98)\n", " model.load_state_dict(torch.load(MODEL_DIRECTORY+'detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pt',map_location=torch.device('cpu')))\n", " model = model.to('cpu')\n", " model.eval()\n", @@ -382,14 +382,11 @@ " # length of domain\n", " df['len'] = df['request_without_domain'].apply(len)\n", " \n", - " # number of subdomains\n", - " df['subdomains_count'] = df['request_without_domain'].apply(lambda x: len(str(x).split('.')))\n", - " \n", " # entropy\n", " df['entropy'] = df['request_without_domain'].apply(lambda x: entropy(x))\n", " \n", " # take most-recent request\n", - " recent_df = df.loc[df['count'] == 1]\n", + " recent_df = df.loc[df['rank'] == 1]\n", "\n", " # calculate feature by aggregating events\n", "\n", @@ -401,10 +398,10 @@ " \n", "# apply model on processed dataframe to predict exfiltration\n", "def apply(model,df,param):\n", - " df.drop(['_time'], axis=1,inplace=True)\n", + " df.drop(['_time'], axis=1,inplace=True, errors='ignore')\n", " recent_df = prepare_input_df(df)\n", - " input_df = recent_df.drop(['src' ,'query','count','request_without_domain','tld'], axis=1)\n", - " recent_df.drop(['request_without_domain','tld','len','subdomains_count','entropy','size_avg','entropy_avg'], axis=1, inplace=True)\n", + " input_df = recent_df.drop(['src' ,'query','rank','request_without_domain','tld'], axis=1)\n", + " recent_df.drop(['request_without_domain','tld','len','entropy','size_avg','entropy_avg'], axis=1, inplace=True)\n", " recent_df.drop(range(0, 94),axis=1,inplace=True)\n", " input_tensor = torch.FloatTensor(input_df.values)\n", " dataloader = DataLoader(input_tensor, shuffle=True, batch_size=256)\n", @@ -416,7 +413,7 @@ " text_rows.clear()\n", " size_avg.clear()\n", " entropy_avg.clear()\n", - " output = pd.merge(recent_df,df,on=['src','query','count'],how='right')\n", + " output = pd.merge(recent_df,df,on=['src','query','rank'],how='right')\n", " return output" ] }, @@ -461,7 +458,7 @@ "source": [ "# load model from name in expected convention \"_\"\n", "def load(name):\n", - " model = DNSExfiltration(99)\n", + " model = DNSExfiltration(98)\n", " model.load_state_dict(torch.load(MODEL_DIRECTORY+'detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.pt',map_location=torch.device('cpu')))\n", " model = model.to('cpu')\n", " model.eval()\n", From 4293a508af1ef0494513cde38bcdf5dfd4c852bc Mon Sep 17 00:00:00 2001 From: Bhavin Patel Date: Mon, 12 Jun 2023 14:16:38 -0700 Subject: [PATCH 8/9] Update detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml Triggering CI --- ...t_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml index a0056b166f..f00144fe98 100644 --- a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml +++ b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -1,5 +1,5 @@ name: Detect DNS Data Exfiltration using pretrained model in DSDL -id: 92f65c3a-968c-11ed-a1eb-0242ac120012 +id: 92f65c3a-168c-11ed-a1eb-0242ac120012 version: 1 date: '2023-04-27' status: experimental @@ -91,4 +91,4 @@ tags: - DNS.dest - DNS.answer risk_score: 45 - security_domain: network \ No newline at end of file + security_domain: network From bc1871025aabaf13bfba854f60a7f98060c30ee9 Mon Sep 17 00:00:00 2001 From: Bhavin Patel Date: Mon, 12 Jun 2023 14:41:43 -0700 Subject: [PATCH 9/9] Update detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml --- ...ect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml index f00144fe98..98e6d33b88 100644 --- a/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml +++ b/detections/network/detect_dns_data_exfiltration_using_pretrained_model_in_dsdl.yml @@ -1,5 +1,5 @@ name: Detect DNS Data Exfiltration using pretrained model in DSDL -id: 92f65c3a-168c-11ed-a1eb-0242ac120012 +id: 92f65c3a-168c-11ed-71eb-0242ac120012 version: 1 date: '2023-04-27' status: experimental