Commit Graph

230 Commits

Author SHA1 Message Date
Will 73d1771205 Restricted two input queues to 1 doc at a time
-Restricted input queue for NLP container to 1 doc at a time
-Restricted input queue for file ingest in enrichment to 1 doc at a time
2024-02-01 14:07:47 -08:00
Will 3f81957c33 Added document average embedding vector indexing
-Added document average embedding vector indexing
2024-02-01 14:06:16 -08:00
Will dabbe94b8b Merge pull request #39 from SpecterOps/tensorflow-removal
Removed tensorflow-serving password model
2024-02-01 12:11:34 -08:00
Will 416f7642ed Removed tensorflow-serving password model
-Removed tensorflow-serving password model as it proved not effective in production.
-Remove classic NLP summarization as it wasn't useful in production
2024-02-01 12:11:12 -08:00
Max Harley 17970d4b5a unbundle from extension-cli (#38) 2024-02-01 12:52:14 -05:00
Will 25743a5ae5 Batching for text vectorization in nlp container
-Batching for text vectorization in `nlp` container
2024-01-31 22:32:40 -08:00
Will ecc6ab8287 Bump for urllib3 for dashboard
-Bump for urllib3 for dashboard
2024-01-31 11:28:50 -08:00
Will b2c0d61a29 Bump for requests in stage1 connector
-Bump for requests in stage1 connector
2024-01-31 11:27:28 -08:00
Will d2f007ac1f Bump requests library for mythic connector
Bump requests library for mythic connector
2024-01-31 11:24:04 -08:00
Will 6a3d8faa9b Bump aiohttp for mythic and sliver connectors
-Bump aiohttp for mythic and sliver connectors
2024-01-31 11:22:47 -08:00
Will 05c088f485 Merge pull request #37 from SpecterOps/dependabot-fixes
Dependabot fixes
2024-01-31 11:19:33 -08:00
Will cc0ebb89e5 Bumped cryptography package in nemesiscommon
-Bumped cryptography package in nemesiscommon
2024-01-31 11:18:35 -08:00
Will 098e6c496f Bumped pycryptodomex package in nemesiscommon
-Bumped pycryptodomex package in nemesiscommon
2024-01-31 11:17:01 -08:00
Will 955d7935b0 fastapi/aiohttp bump and remove dependency for fastapi-class for enrichment
-fastapi/aiohttp bump and remove dependency for fastapi-class for `enrichment`
2024-01-31 11:11:48 -08:00
Will cd7655e1db Bump fastapi and aiohttp for passwordcracker
-Bump fastapi and aiohttp for `passwordcracker`
2024-01-30 18:02:07 -08:00
Will 419a4ca6f7 fastapi bump and remove dependency for fastapi-class for nlp
-fastapi bump and remove dependency for fastapi-class for `nlp`
-bump aiohttp
2024-01-30 18:01:45 -08:00
Will c993c68b7f Updated poetry.lock for nemesiscommon to update aiohttp depdendency
-Updated poetry.lock for nemesiscommon to update aiohttp depdendency
2024-01-30 17:45:09 -08:00
Will 78a1554632 fastapi bump and remove dependency for fastapi-class
-fastapi bump and remove dependency for fastapi-class
2024-01-30 16:47:09 -08:00
Will 186b494e55 Bump aiohttp package in dotnet container
-Bump aiohttp package in `dotnet` container
2024-01-30 15:29:45 -08:00
Will 154a21f437 Handle parsing of additional ls format in stage1 connector
-Handle parsing of additional ls format in stage1 connector
2024-01-29 16:06:59 -08:00
Will 9c397dfd02 Update Hasura schema
-Update Hasura schema
2024-01-29 14:59:12 -08:00
Will 9ce864de00 Add support for Merlin's ls tasks in mythic-connector
-Add support for Merlin's ls tasks in `mythic-connector`
2024-01-29 14:58:58 -08:00
Will 01bc999b60 Handle cwd response for stage1 connector
-Handle cwd response for stage1 connector
-Sleep on startup so other services can start
2024-01-26 13:03:42 -08:00
Will d8b1dd5af3 for stage1 connector, auto task pwd if one doesn't exist
-for stage1 connector, auto task pwd if one doesn't exist
2024-01-25 16:01:59 -08:00
Will 718bae8b8b Stage1 connector work
-Implemented syncing of file listings
-Implemented reprocessing of downloads and file listings
2024-01-25 15:48:43 -08:00
Will a19b915d8f Small dashboard fixes
-Fix for when to display File Canaries tab
-Fix for Dashboard streamlit app for file monitoring ("inotify instance limit reached" error)
2024-01-24 13:48:51 -08:00
Will ffd729936d Merge pull request #36 from SpecterOps/hasura-initial
Added Hasura endpoint with schema initialization
2024-01-22 15:55:10 -08:00
Will ce27603cbb Added Hasura with schema initialization
-Added Hasura with schema initialization
2024-01-22 15:54:26 -08:00
Will 45f61c3c9e Merge pull request #35 from SpecterOps/crack-list_mods
Crack-List mods
2024-01-22 12:54:35 -08:00
Will c0fc6b85b4 Crack-List mods
-Added page on the Dashboard to download custom cracklist
-Minor cleanup of cracklist code
-Added persistent storage for cracklists for non-dev
2024-01-22 12:53:29 -08:00
Will 7aeeb692aa Don't try to re-crack hashes that didn't crack previously
-Don't try to re-crack hashes that didn't crack previously
2024-01-22 10:46:38 -08:00
Will 31a400c93a Merge branch 'main' of github.com:SpecterOps/Nemesis 2024-01-22 10:24:43 -08:00
Will 2dad913590 Updated langchain to use langchain-community for HuggingFaceEmbeddings
-Updated langchain to use langchain-community for HuggingFaceEmbeddings
2024-01-22 10:16:29 -08:00
Will 3afc422ce2 Merge pull request #34 from SpecterOps/passwordcracker_mods
Passwordcracker mods
2024-01-22 10:15:22 -08:00
Will 8fd06aa7ca Don't re-crack already cracked hashes
-Check if a hash is already present and cracked in the database and use the cracked value instead of re-cracking the hash
2024-01-22 10:14:00 -08:00
Will 81cb8a3267 Uniquness for extracted hashes in DB
-Uniquness for extracted hashes in DB
2024-01-19 18:35:40 -08:00
Will 605d1ff9ce Ingest extracted hash into database immediately
-Ingest extracted hash into database immediately before cracking attempted
-Roll back JTR image to previous format due to speed issues
2024-01-19 15:26:15 -08:00
Will c60d094843 -Updated dotnet Poetry package dependencies with poetry lock
-Updated `dotnet` Poetry package dependencies with `poetry lock`
2024-01-19 13:08:59 -08:00
Will 0faad3a237 Updated passwordcracker Poetry package dependencies
-Updated `passwordcracker` Poetry package dependencies with `poetry lock`
2024-01-19 13:04:34 -08:00
Will 1b468a2f22 Merge pull request #33 from SpecterOps/NLP_container_fixes
NLP container fixes
2024-01-19 12:52:42 -08:00
Will 8d9fdcfc8c Changed Semantic Search result display in Dashboard
-Changed Semantic Search result display in Dashboard
2024-01-19 12:47:57 -08:00
Will 7f34ffd762 Changed embedding model for NLP container
-Changed embedding model for NLP container to one that has a proper context window
-Updated to ElasticsearchStore in NLP app
-No longer collapse all whitespace in indexing
2024-01-19 12:47:34 -08:00
Will 982b86ac79 Bumped versions for dependencies
-Bumped versions for langchain, tiktoken, and pytorch
2024-01-19 12:46:05 -08:00
Will 478192a286 Canary display tab in File Viewer
-Added a canary display tab in File Viewer
-`file_canary` tag in the file listing is now clickable
2024-01-18 13:47:08 -08:00
Will 16308b6e17 UTF-8/UTF-16 strings handling in Yara Matches
-UTF-8/UTF-16 strings handling in Yara Matches
2024-01-18 12:45:42 -08:00
Will 53777ef87e Merge pull request #32 from SpecterOps/yara_mods
More Yara mods
2024-01-18 00:08:24 -08:00
Will 342ca28606 Revamp for Yara match display in File Viewer
-Revamp for Yara match display in File Viewer
-Now includes rule definition and string matches w/ offset + length
2024-01-18 00:05:39 -08:00
Will 67fdfc57a2 Expanded Yara rule context
-Expanded Yara rule context parsed by the file enrichment service, now includes matching strings as well as the rule definition itself
2024-01-17 23:29:10 -08:00
Will 467fc2727c Merge pull request #31 from SpecterOps/yararule-mods
Yara Rule mods
2024-01-17 20:05:05 -08:00
Will f8eb2cfecd Completed Yara Match JSON display tab in FileViewer
-Completed Yara Match JSON display tab in FileViewer
-Added link to the viewer in main file listing
2024-01-17 20:02:13 -08:00