admin/schwartz1375-genai-essentials
1
0
Fork 0
mirror of https://github.com/schwartz1375/genai-essentials synced 2026-08-09 13:08:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
6c257ebab03f53afa4d09011ba263670594b01f4
schwartz1375-genai-essentials/llm_tutorial.ipynb
T
schwartz1375 6c257ebab0 Change embedding model to embeddinggemma and fix ChromaDB initialization 
- Switch from nomic-embed-text to embeddinggemma for embeddings
- Add directory verification cells to ensure required directories exist
- Add error handling for corrupted ChromaDB databases with auto-recovery
- Create persist directories before ChromaDB initialization
2025-12-06 16:59:15 -05:00

1367 lines
99 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Environment Setup\n",
"\n",
"This notebook demonstrates LangChain concepts using **local models via Ollama**. Before continuing, make sure the required models and Python packages are installed.\n",
"\n",
"---\n",
"\n",
"### 1. Install and Configure Ollama\n",
"\n",
"If you haven't already, install [Ollama](https://ollama.com) and ensure it's running locally.\n",
"\n",
"Then pull the required models:\n",
"\n",
"```bash\n",
"ollama pull gemma3\n",
"ollama pull embeddinggemma\n",
"```\n",
"\n",
"> These models will be used for generation (`gemma3`) and embeddings (`embeddinggemma`).\n",
"\n",
"---\n",
"\n",
"### 2. Install Required Python Packages"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'\\n!pip install --upgrade pip --quiet\\n!pip install -U langchain langchain-ollama langchain-community langchain-core langchain-experimental chromadb pandas pymupdf ipython duckduckgo-search --quiet\\n'"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"'''\n",
"!pip install --upgrade pip --quiet\n",
"!pip install -U \\\n",
" langchain langchain-ollama langchain-community langchain-core \\\n",
" langchain-experimental chromadb pandas pymupdf ipython \\\n",
" duckduckgo-search --quiet\n",
"'''"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3. Start Ollama (if not already running)\n",
"\n",
"Make sure Ollama is running in the background. You can check by running:\n",
"\n",
"```bash\n",
"ollama list\n",
"```\n",
"\n",
"This should list both `gemma3:latest` and `embeddinggemma` models."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Local vs. Cloud Models\n",
"\n",
"### Ollama (Local Execution)\n",
"\n",
"Ollama allows you to run open-source models (e.g., `gemma`, `llama`, `embeddinggemma`) locally. This provides control, privacy, and fast iteration without API latency or cost.\n",
"\n",
"### Amazon Bedrock (Cloud Execution)\n",
"\n",
"[Amazon Bedrock](https://aws.amazon.com/bedrock/) provides access to multiple foundation models through a single API. It is fully managed, scalable, and integrates into AWS’s security and governance ecosystem.\n",
"\n",
"## Basic Concepts of Generative AI\n",
"\n",
"This section introduces core concepts for working with Large Language Models (LLMs), including how to prompt them effectively and build applications using frameworks like **LangChain**—whether deploying models locally via **Ollama** or through cloud services like **AWS Bedrock**.\n",
"\n",
"---\n",
"\n",
"### What Are LLMs?\n",
"\n",
"**Large Language Models (LLMs)** are machine learning models trained on massive text corpora. They predict the next token in a sequence based on context, making them powerful pattern-recognition tools for generating human-like text. However, they do not understand meaning or possess reasoning capabilities—their output is probabilistic, not factual by default.\n",
"\n",
"---\n",
"\n",
"### Prompting and Prompt Engineering\n",
"\n",
"**Prompts** are structured inputs that tell an LLM what to do. A well-crafted prompt can significantly influence the quality, accuracy, and usefulness of the model's response.\n",
"\n",
"Prompts generally consist of three key components:\n",
"\n",
"---\n",
"\n",
"#### Instruction\n",
"\n",
"The **instruction** specifies the task or behavior expected from the model. It can be a simple question, a directive, or even a persona setup.\n",
"\n",
"**Examples:**\n",
"- “Generate a list of known malicious domains associated with the Qakbot malware family.”\n",
"- “Write a poem about the Qakbot malware family.”\n",
"- “Produce a sonnet about my love of Franklin’s BBQ.”\n",
"\n",
"---\n",
"\n",
"#### Context\n",
"\n",
"**Context** includes relevant information provided to the model to help it complete the instruction. This is particularly important for specialized or domain-specific use cases.\n",
"\n",
"**Examples of context:**\n",
"- Extracted text from a PDF or website\n",
"- Lists of known facts or structured datasets\n",
"- Internal business knowledge\n",
"\n",
"Context is critical when the model lacks the necessary background knowledge. It also enables **in-context learning** and is foundational to **RAG (Retrieval-Augmented Generation)** systems.\n",
"\n",
"**Reference:** \n",
"- [AWS Blog: Context Window Overflow – Breaking the Barrier](https://aws.amazon.com/blogs/security/context-window-overflow-breaking-the-barrier/)\n",
"\n",
"---\n",
"\n",
"#### Output Format\n",
"\n",
"Defining the **output format** ensures the model returns a response in a structure that's compatible with downstream processing or presentation.\n",
"\n",
"**Examples:**\n",
"- JSON/JSONL for programmatic consumption\n",
"- Markdown for documentation or emails\n",
"- Lists, CSV, or tabular formats for parsing\n",
"\n",
"Providing formatting examples in your prompt often leads to more consistent and usable output.\n",
"\n",
"---\n",
"\n",
"#### Best Practices for Prompt Engineering\n",
"\n",
"- Be explicit and unambiguous in your instructions.\n",
"- Focus on what you want, not just what to avoid.\n",
"- Supply sufficient context for the model to reason accurately.\n",
"- Specify format, tone, language, or response length as needed.\n",
"- Iterate and refine—prompt design is an interactive process.\n",
"- Be cost-conscious—longer prompts and responses mean higher token usage.\n",
"- Leverage in-context examples to teach the model new behavior.\n",
"\n",
"**Prompting strategies:**\n",
"- **Zero-shot**: No examples provided.\n",
"- **One-shot**: One example included.\n",
"- **Few-shot**: Multiple examples included.\n",
"\n",
"---\n",
"\n",
"### System Prompts\n",
"\n",
"Some platforms (e.g., OpenAI, Anthropic) support **system prompts**—instructions that shape the model’s persona, tone, or behavioral constraints.\n",
"\n",
"**Example system message:** \n",
"> “You are a cybersecurity analyst generating threat intelligence summaries.”\n",
"\n",
"System prompts are not typically hidden; providers often document their usage:\n",
"\n",
"- [Anthropic: System Prompt Guidelines](https://docs.anthropic.com/en/release-notes/system-prompts)\n",
"\n",
"---\n",
"\n",
"### Fine-Tuning vs. In-Context Learning\n",
"\n",
"- **Fine-tuning** modifies the model’s internal weights using domain-specific data. It’s powerful but resource-intensive and harder to maintain.\n",
"- **In-context learning** gives examples inline in the prompt. It’s flexible, efficient, and sufficient for many real-world use cases.\n",
"\n",
"---\n",
"\n",
"**Learn more:** [Prompt Engineering Guide](https://www.promptingguide.ai/)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Core LangChain Concepts\n",
"\n",
"[LangChain](https://python.langchain.com/) is a modular framework for building generative AI applications powered by Large Language Models (LLMs). It simplifies the development of multi-step pipelines, retrieval-augmented workflows, and agent-based reasoning systems.\n",
"\n",
"---\n",
"\n",
"### LangChain Capabilities\n",
"\n",
"LangChain provides key building blocks for LLM applications:\n",
"\n",
"- **Prompt templates** for reusability and structure\n",
"- **Memory** for multi-turn interactions\n",
"- **Tools** to access external data sources\n",
"- **Chains** to compose tasks in sequence or branching logic\n",
"- **Agents** for dynamic tool use and reasoning\n",
"- **Document loaders, chunkers, and retrievers** for knowledge ingestion\n",
"\n",
"---\n",
"\n",
"### Prompt Templates\n",
"\n",
"**Prompt templates** allow you to define reusable prompts with variables. This is essential for consistent prompt formatting in production use cases.\n",
"\n",
"```python\n",
"from langchain.prompts import PromptTemplate\n",
"\n",
"template = \"Generate a list of domains associated with {malware_family}\"\n",
"prompt = PromptTemplate.from_template(template)\n",
"prompt.format(malware_family=\"Qakbot\")\n",
"```\n",
"\n",
"---\n",
"\n",
"### Memory\n",
"\n",
"LLMs are inherently **stateless**, meaning they do not retain context between interactions. LangChain provides memory modules such as `ConversationBufferMemory` that persist prior messages and automatically re-inject them into prompts.\n",
"\n",
"```python\n",
"from langchain.memory import ConversationBufferMemory\n",
"\n",
"memory = ConversationBufferMemory()\n",
"```\n",
"\n",
"When used in a chain:\n",
"\n",
"```python\n",
"chain = LLMChain(llm=llm, prompt=prompt_template, memory=memory)\n",
"```\n",
"\n",
"Memory is critical for applications like chatbots, copilots, or multi-turn reasoning agents.\n",
"\n",
"---\n",
"\n",
"### Tools\n",
"\n",
"**Tools** allow LLMs to interact with external systems—APIs, web search engines, internal databases, or third-party services. LangChain supports tool integration via descriptors that let agents reason about what tool to use and when.\n",
"\n",
"**Examples of tools:**\n",
"- Google/Bing search\n",
"- DomainTools, VirusTotal, Wikipedia\n",
"- Custom internal APIs (e.g., threat intelligence, ticketing systems)\n",
"\n",
"More: [LangChain Tools Documentation](https://python.langchain.com/docs/modules/tools/)\n",
"\n",
"---\n",
"\n",
"### Chains\n",
"\n",
"**Chains** are workflows composed of modular steps. Each chain links one or more components (e.g., prompt → LLM → post-processor).\n",
"\n",
"Common chain types include:\n",
"\n",
"- `LLMChain`: single prompt-response (Note: LLMChain is deprecated in favor of LCEL - LangChain Expression Language)\n",
"- `SimpleSequentialChain`: sequential chaining of steps\n",
"- `MultiInputChain`: accepts multiple inputs\n",
"- `RouterChain`: dynamically selects downstream chains based on input\n",
"\n",
"Example with memory:\n",
"\n",
"```python\n",
"chain = LLMChain(llm=llm, prompt=prompt_template, memory=ConversationBufferMemory())\n",
"```\n",
"\n",
"Chains provide structure and allow you to build reliable pipelines with conditional logic and state.\n",
"\n",
"---\n",
"\n",
"### Agents\n",
"\n",
"**Agents** enable dynamic, reasoning-driven workflows. Unlike static chains, agents interpret user input, determine which tools or sub-chains to invoke, gather additional data, and synthesize final responses.\n",
"\n",
"They excel at handling ambiguous prompts, complex logic, or open-ended tasks.\n",
"\n",
"More: [LangChain Agents Documentation](https://python.langchain.com/docs/modules/agents/)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## RAG and Embeddings\n",
"\n",
"### Vector Stores\n",
"\n",
"Vector databases (e.g., Chroma, FAISS, Pinecone) store **text embeddings**, which allow similarity searches against known documents or facts.\n",
"\n",
"### RAG (Retrieval-Augmented Generation)\n",
"\n",
"**RAG** improves response quality by retrieving relevant content from an external knowledge source before prompting the model. This enables high-accuracy answers even from models that weren’t explicitly trained on the content.\n",
"\n",
"### Chunking\n",
"\n",
"Long documents are split into smaller **chunks** to improve retrieval accuracy and avoid exceeding token limits. LangChain includes text splitters like `RecursiveCharacterTextSplitter` that optimize chunk size and overlap.\n",
"\n",
"### Embeddings\n",
"\n",
"Embeddings are numeric vectors that represent semantic meaning. They are used for similarity search in vector stores.\n",
"\n",
"- **Local**: e.g., `embeddinggemma` via Ollama\n",
"- **Cloud**: e.g., Titan via AWS Bedrock, HuggingFace SentenceTransformers\n",
"\n",
"---\n",
"\n",
"### LLM Limitations\n",
"\n",
"- **Hallucination**: Models may fabricate plausible but incorrect information.\n",
"- **Context limitations**: Only a fixed number of tokens can be processed per prompt.\n",
"- **Data leakage risk**: LLMs trained on sensitive data may inadvertently reveal it.\n",
"- **Stateless**: Models forget previous interactions unless provided memory mechanisms.\n",
"\n",
"---\n",
"\n",
"### Understanding Tokenization\n",
"\n",
"LLMs process input/output as **tokens**, not words. Token count affects:\n",
"\n",
"- Cost (in usage-based billing models)\n",
"- Whether your full prompt fits in the model’s context window\n",
"- Output truncation risk\n",
"\n",
"Useful tools:\n",
"- [Claude Tokenizer](https://claude-tokenizer.vercel.app/)\n",
"- [TikToken Tokenizer](https://tiktikenizer.vercel.app/)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Calling Ollama via LangChain - Zero Shot\n",
"https://python.langchain.com/api_reference/ollama/chat_models/langchain_ollama.chat_models.ChatOllama.html"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"That's a fantastic question! The blue color of the sky is a really cool phenomenon, and it's all thanks to something called **Rayleigh scattering**. Here’s a breakdown of how it works:\n",
"\n",
"**1. Sunlight and Colors:**\n",
"\n",
"* Sunlight isn't actually white. It's made up of *all* the colors of the rainbow – red, orange, yellow, green, blue, indigo, and violet. Think about a prism splitting sunlight.\n",
"\n",
"**2. Scattering of Light:**\n",
"\n",
"* As sunlight enters the Earth's atmosphere, it collides with tiny air molecules (mostly nitrogen and oxygen).\n",
"* This collision causes the light to scatter in different directions. This scattering is what makes the sky appear blue.\n",
"\n",
"**3. Rayleigh Scattering – The Key:**\n",
"\n",
"* **Rayleigh scattering** is *more effective* at scattering shorter wavelengths of light. Blue and violet light have shorter wavelengths than other colors like red and orange.\n",
"* Because blue and violet light are scattered much more strongly, they bounce around in all directions throughout the atmosphere.\n",
"\n",
"**4. Why Not Violet?**\n",
"\n",
"* You might wonder, “If violet light scatters even *more* than blue, why isn't the sky"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from IPython.display import Markdown\n",
"from langchain_ollama import ChatOllama \n",
"\n",
"# Note: Ensure Ollama is running locally and the gemma3 model is installed\n",
"# Run: ollama pull gemma3\n",
"# If Ollama is not running, you'll get a connection error\n",
"\n",
"llm = ChatOllama(\n",
" model=\"gemma3\",\n",
" temperature=0.8,\n",
" # base_url=\"http://remote-ip:port\", # Ollama default port: 11434\n",
" # timeout=5,\n",
" num_predict=256, # Use num_predict instead of max_tokens for Ollama\n",
" # other params ...\n",
")\n",
"\n",
"messages = [\n",
" (\"system\", \"You are a helpful assistant.\"),\n",
" (\"human\", \"why is the sky blue?\"),\n",
"]\n",
"\n",
"Markdown(llm.invoke(messages).content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using Prompt Templates and LangChain Expression Syntax\n",
"\n",
"This example introduces the use of a `ChatPromptTemplate` combined with an Ollama-backed LLM using LangChain's expression syntax. This pattern allows for reusable, parameterized prompts and demonstrates how to integrate structured prompting into your pipeline. The `|` operator chains the prompt and model together, creating a modular and declarative workflow.\n",
"\n",
"We’ll also simulate a multi-role setup (e.g., system + human) within the prompt string.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"Okay, let’s break down LummaC2.\n",
"\n",
"**LummaC2: A Sophisticated, Multi-Stage Cyber-Espionage Tool**\n",
"\n",
"LummaC2 (formerly known as “TRASH”) is a highly advanced, modular cyber-espionage tool developed by the Russian-linked APT group known as “Vandyr.” It’s significantly more complex than many common malware families, making it a particularly concerning threat. Here’s a breakdown of what makes it notable:\n",
"\n",
"**1. Modular Architecture:** This is the key differentiator. Unlike many malware families that are single, monolithic programs, LummaC2 is built around a modular design. This means it’s comprised of several distinct components, each performing a specific function. These modules work together to achieve the overall goal of espionage. This modularity provides several advantages to the attackers:\n",
"\n",
" * **Flexibility:** Attackers can easily swap out modules, allowing them to quickly adapt to new environments and targets.\n",
" * **Stealth:** The modular design makes it harder for security analysts to identify and understand the entire scope of the attack.\n",
" * **Persistence:** Multiple modules can ensure the malware remains active even if one component is detected and removed.\n",
"\n",
"\n",
"**2. Stages of Operation (Typical Lifecycle):** LummaC2 generally operates through several stages:\n",
"\n",
" * **Initial Access:** This could involve phishing emails, compromised websites, or exploiting vulnerabilities in software. They have been observed utilizing various techniques.\n",
" * **Beaconing:** Once inside a network, LummaC2 establishes a persistent connection (a \"beacon\") back to the command-and-control (C2) server. This beacon allows the attackers to remotely control the infected system.\n",
" * **Data Exfiltration:** The core espionage function. LummaC2 is designed to steal sensitive data. It targets specific data based on the targets' profiles – government agencies, defense contractors, and critical infrastructure are frequently targeted.\n",
" * **Credential Theft:** LummaC2 actively attempts to steal credentials (usernames, passwords, etc.) to maintain access and escalate privileges.\n",
" * **Lateral Movement:** After establishing initial access, LummaC2 attempts to move laterally through the network to gain access to more systems and data.\n",
"\n",
"\n",
"**3. Key Features & Technical Details:**\n",
"\n",
" * **Multi-Protocol C2:** It uses multiple protocols (HTTP, HTTPS, DNS) for communication with its C2 server. This makes it harder to detect by blocking only one protocol.\n",
" * **Dynamic DNS:** To avoid being blocked, LummaC2 uses dynamic DNS services to ensure the C2 server’s address remains consistent.\n",
" * **PowerShell-based Modules:** A significant portion of its functionality relies on PowerShell, which allows for deep integration with Windows systems.\n",
" * **Anti-Analysis Techniques:** LummaC2 employs techniques to evade detection during analysis, such as process injection and obfuscation.\n",
"\n",
"**4. Threat Intelligence & Targets:**\n",
"\n",
" * **Russian-linked APT:** Almost exclusively attributed to Vandyr, a group often associated with Russian intelligence services.\n",
" * **High-Value Targets:** Primarily focuses on government, defense, and critical infrastructure sectors. Specific targets have included defense contractors and organizations dealing with sensitive information.\n",
"\n",
"\n",
"\n",
"**Resources for Further Research:**\n",
"\n",
"* **CrowdStrike Report:** [https://www.crowdstrike.com/blog/lumma-c2-ransomware-variant-or-cyber-espionage-tool/](https://www.crowdstrike.com/blog/lumma-c2-ransomware-variant-or-cyber-espionage-tool/) (CrowdStrike’s detailed analysis)\n",
"* **Unit 42 Blog:** [https://unit42.paloaltonetworks.com/lumma-c2-cyber-espionage-tool/](https://www.unit42.paloaltonetworks.com/lumma-c2-cyber-espionage-tool/) (Palo Alto Networks’ analysis)\n",
"\n",
"\n",
"Do you want me to delve deeper into a specific aspect of LummaC2, such as:\n",
"\n",
"* Its C2 communication protocols?\n",
"* Specific techniques used to evade detection?\n",
"* How it compares to other cyber-espionage tools?"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_core.prompts import ChatPromptTemplate\n",
"from langchain_ollama.llms import OllamaLLM\n",
"\n",
"template = \"\"\"Question: {question}\n",
"\n",
"Answer: Let's think step by step.\"\"\"\n",
"\n",
"prompt = ChatPromptTemplate.from_template(template)\n",
"\n",
"model = OllamaLLM(model=\"gemma3\")\n",
"\n",
"chain = prompt | model\n",
"\n",
"Markdown(chain.invoke({\"question\": \"System: You are cybersecurity expert, and AI assisant to AWS.\\n\\nHuman: What is the LummaC2 malware?\"}))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Memory\n",
"\n",
"It's nice to be able to ask a question and get an answer, but that seems pretty transactional and impersonal. I'd like the LLM I'm talking to to show me they are listening to me and paying attention to what I am saying. I am, after all, human and I need to be loved or at least feel like it.\n",
"\n",
"But LLMs are cold and heartless (technically, they are stateless but we are talking about my feelings here). This means that they don’t ‘remember’ interactions from prompt to prompt. You can fine tune them to persist data but without updating the weights through expensive training but, they don’t ‘remember’ anything more after the model is set at a checkpoint. This means if I just asked what the Qakbot malware was and then follow up with a question like \"What industry was primarily targeted by this malware?\", the model will not be able to answer within the context of Qakbot...since show beats tell, let's see how that works firsthand."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Now lets ask the followup question..."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"Please provide me with the details about the malware you’re referring to. I need information like:\n",
"\n",
"* **The name of the malware:** (e.g., WannaCry, NotPetya, Emotet, etc.)\n",
"* **A description of its behavior:** (e.g., ransomware, botnet, phishing tool, etc.)\n",
"* **Any known indicators of compromise (IOCs):** (e.g., IP addresses, domain names, file hashes)\n",
"* **Any reports or analysis you have about it.** \n",
"\n",
"Once I have this information, I can tell you which industry was primarily targeted. \n",
"\n",
"**Without this information, I can’t answer your question.**"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Markdown(llm.invoke(\"System: You are cybersecurity expert, and AI assisant.\\n\\nHuman: What industry was primarily targeted by this malware?\").content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## LangChain Memory Components: Enabling Stateful Conversations\n",
"The memory component of LangChain allows the LLM to become 'stateful'. This quality is quite useful when developing applications driven by LLMs. For instance, a conversational system or a chatbot is required to recall past interactions to maintain a conversation. Without memory, the system would not be able to handle follow-up messages or recollect key pieces of information mentioned earlier in the conversation. \n",
"\n",
"In this section, we will explore the memory modules provided by LangChain. LangChain offers several types of memory modules depending on the task and the properties of the LLM. We will incorporate memory into chains and examine changes in performance due to it."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"from langchain_community.llms import Ollama\n",
"from langchain.chains import ConversationChain\n",
"from langchain.memory import ConversationBufferMemory\n",
"import pandas as pd\n",
"import warnings\n",
"\n",
"warnings.filterwarnings(\"ignore\")\n",
"\n",
"# Initialize Ollama with Gemma model\n",
"llm = Ollama(model=\"gemma3\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.microsoft.datawrangler.viewer.v0+json": {
"columns": [
{
"name": "index",
"rawType": "int64",
"type": "integer"
},
{
"name": "input",
"rawType": "object",
"type": "string"
},
{
"name": "history",
"rawType": "object",
"type": "unknown"
},
{
"name": "response",
"rawType": "object",
"type": "string"
}
],
"ref": "ce04f3a5-19a6-42c4-8dcd-3b620ceafb72",
"rows": [
[
"0",
"What is the LummaC2 malware?",
"{'history': \"Human: What is the LummaC2 malware?\\nAI: Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\\n\\nHere’s a breakdown of what makes it notable:\\n\\n* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\\n\\n* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\\n * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\\n * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\\n * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\\n * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\\n * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\\n\\n* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\\n\\n* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\\n\\n* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\\n\\nDo you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group?\"}",
"Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\n\nHere’s a breakdown of what makes it notable:\n\n* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\n\n* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\n * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\n * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\n * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\n * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\n * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\n\n* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\n\n* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\n\n* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\n\nDo you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group?"
],
[
"1",
"What industry was primarily targeted by this malware?",
"{'history': \"Human: What is the LummaC2 malware?\\nAI: Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\\n\\nHere’s a breakdown of what makes it notable:\\n\\n* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\\n\\n* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\\n * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\\n * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\\n * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\\n * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\\n * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\\n\\n* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\\n\\n* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\\n\\n* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\\n\\nDo you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group?\\nHuman: What industry was primarily targeted by this malware?\\nAI: Okay, great question! While LummaC2 has been observed targeting a *range* of industries, the primary targets have consistently been the defense, energy, and technology sectors. As I mentioned before, Sophos has specifically noted a strong focus on companies with a global presence within those industries. They’ve seen a significant number of campaigns directed at organizations involved in defense technology – think things like missile defense systems or cybersecurity products – and companies involved in energy production and distribution. It's not that other industries weren't *ever* targeted, but those three were the most frequent and impactful.\\n\\nDo you want me to elaborate on *why* these sectors were particularly attractive to UNC3923, or perhaps discuss the specific types of data they were seeking within those industries? Or would you like me to go back and explain a particular module in more detail, like the lateral movement capabilities?\"}",
"Okay, great question! While LummaC2 has been observed targeting a *range* of industries, the primary targets have consistently been the defense, energy, and technology sectors. As I mentioned before, Sophos has specifically noted a strong focus on companies with a global presence within those industries. They’ve seen a significant number of campaigns directed at organizations involved in defense technology – think things like missile defense systems or cybersecurity products – and companies involved in energy production and distribution. It's not that other industries weren't *ever* targeted, but those three were the most frequent and impactful.\n\nDo you want me to elaborate on *why* these sectors were particularly attractive to UNC3923, or perhaps discuss the specific types of data they were seeking within those industries? Or would you like me to go back and explain a particular module in more detail, like the lateral movement capabilities?"
]
],
"shape": {
"columns": 3,
"rows": 2
}
},
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>input</th>\n",
" <th>history</th>\n",
" <th>response</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>What is the LummaC2 malware?</td>\n",
" <td>{'history': 'Human: What is the LummaC2 malware?\n",
"AI: Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\n",
"\n",
"Here’s a breakdown of what makes it notable:\n",
"\n",
"* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\n",
"\n",
"* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\n",
" * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\n",
" * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\n",
" * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\n",
" * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\n",
" * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\n",
"\n",
"* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\n",
"\n",
"* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\n",
"\n",
"* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\n",
"\n",
"Do you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group?'}</td>\n",
" <td>Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\\n\\nHere’s a breakdown of what makes it notable:\\n\\n* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\\n\\n* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\\n * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\\n * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\\n * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\\n * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\\n * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\\n\\n* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\\n\\n* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\\n\\n* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\\n\\nDo you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group?</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>What industry was primarily targeted by this malware?</td>\n",
" <td>{'history': 'Human: What is the LummaC2 malware?\n",
"AI: Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\n",
"\n",
"Here’s a breakdown of what makes it notable:\n",
"\n",
"* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\n",
"\n",
"* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\n",
" * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\n",
" * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\n",
" * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\n",
" * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\n",
" * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\n",
"\n",
"* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\n",
"\n",
"* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\n",
"\n",
"* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\n",
"\n",
"Do you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group?\n",
"Human: What industry was primarily targeted by this malware?\n",
"AI: Okay, great question! While LummaC2 has been observed targeting a *range* of industries, the primary targets have consistently been the defense, energy, and technology sectors. As I mentioned before, Sophos has specifically noted a strong focus on companies with a global presence within those industries. They’ve seen a significant number of campaigns directed at organizations involved in defense technology – think things like missile defense systems or cybersecurity products – and companies involved in energy production and distribution. It's not that other industries weren't *ever* targeted, but those three were the most frequent and impactful.\n",
"\n",
"Do you want me to elaborate on *why* these sectors were particularly attractive to UNC3923, or perhaps discuss the specific types of data they were seeking within those industries? Or would you like me to go back and explain a particular module in more detail, like the lateral movement capabilities?'}</td>\n",
" <td>Okay, great question! While LummaC2 has been observed targeting a *range* of industries, the primary targets have consistently been the defense, energy, and technology sectors. As I mentioned before, Sophos has specifically noted a strong focus on companies with a global presence within those industries. They’ve seen a significant number of campaigns directed at organizations involved in defense technology – think things like missile defense systems or cybersecurity products – and companies involved in energy production and distribution. It's not that other industries weren't *ever* targeted, but those three were the most frequent and impactful.\\n\\nDo you want me to elaborate on *why* these sectors were particularly attractive to UNC3923, or perhaps discuss the specific types of data they were seeking within those industries? Or would you like me to go back and explain a particular module in more detail, like the lateral movement capabilities?</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" input \\\n",
"0 What is the LummaC2 malware? \n",
"1 What industry was primarily targeted by this malware? \n",
"\n",
" history \\\n",
"0 {'history': 'Human: What is the LummaC2 malware?\n",
"AI: Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\n",
"\n",
"Here’s a breakdown of what makes it notable:\n",
"\n",
"* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\n",
"\n",
"* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\n",
" * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\n",
" * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\n",
" * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\n",
" * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\n",
" * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\n",
"\n",
"* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\n",
"\n",
"* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\n",
"\n",
"* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\n",
"\n",
"Do you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group?'} \n",
"1 {'history': 'Human: What is the LummaC2 malware?\n",
"AI: Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\n",
"\n",
"Here’s a breakdown of what makes it notable:\n",
"\n",
"* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\n",
"\n",
"* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\n",
" * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\n",
" * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\n",
" * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\n",
" * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\n",
" * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\n",
"\n",
"* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\n",
"\n",
"* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\n",
"\n",
"* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\n",
"\n",
"Do you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group?\n",
"Human: What industry was primarily targeted by this malware?\n",
"AI: Okay, great question! While LummaC2 has been observed targeting a *range* of industries, the primary targets have consistently been the defense, energy, and technology sectors. As I mentioned before, Sophos has specifically noted a strong focus on companies with a global presence within those industries. They’ve seen a significant number of campaigns directed at organizations involved in defense technology – think things like missile defense systems or cybersecurity products – and companies involved in energy production and distribution. It's not that other industries weren't *ever* targeted, but those three were the most frequent and impactful.\n",
"\n",
"Do you want me to elaborate on *why* these sectors were particularly attractive to UNC3923, or perhaps discuss the specific types of data they were seeking within those industries? Or would you like me to go back and explain a particular module in more detail, like the lateral movement capabilities?'} \n",
"\n",
" response \n",
"0 Okay, let’s talk about LummaC2! It’s a really interesting and quite sophisticated piece of malware that’s been tracked by analysts at Sophos. Essentially, it's a modular, multi-stage post-exploitation framework primarily targeting Windows systems.\\n\\nHere’s a breakdown of what makes it notable:\\n\\n* **Developed by the Chinese APT group, UNC3923:** Sophos has linked it very strongly to this group, which is known for its espionage activities, often targeting the energy, defense, and technology sectors. They’ve been tracking this group for a *long* time – back to 2018 – and LummaC2 is a key component of their toolkit.\\n\\n* **Modular Design:** This is a *big* deal. LummaC2 isn't just one piece of malware; it’s composed of several modules that can be used individually or combined. This makes it highly adaptable and difficult to detect. The main modules include:\\n * **Beacon:** This is the core module. It’s a C2 (Command and Control) beacon that allows the attackers to communicate with the infected host. It uses TLS encryption for secure communication – they’re not sending data in the clear. The beacon itself utilizes multiple protocols – HTTP, HTTPS, and even DNS – to evade detection.\\n * **Credential Dumping:** This module attempts to steal credentials (usernames, passwords, hashes) from the infected system. They've been observed using tools like Mimikatz – a very common tool for credential theft.\\n * **Process Injection:** They use this to inject malicious code into legitimate running processes, further expanding their control over the system.\\n * **Privilege Escalation:** LummaC2 attempts to escalate privileges on the infected system, allowing them to gain higher levels of control.\\n * **Lateral Movement:** This is crucial. Once they have control of one machine, they use it to move laterally across the network, compromising other systems. They’ve been observed using tools like PsExec for this.\\n\\n* **TLS Encryption:** As I mentioned before, the beacon uses TLS (Transport Layer Security) encryption. This makes it harder for security analysts to monitor the communication and understand what the malware is doing in real-time.\\n\\n* **Observed Targets:** They’ve been actively targeting companies in the defense, energy, and technology sectors – specifically those with a global presence.\\n\\n* **Sophos Tracking:** Sophos has been diligently tracking LummaC2's activity since 2021, publishing detailed reports on its behavior and associated IOCs (Indicators of Compromise) – like IP addresses, domains, and file hashes – that security teams can use to detect and block the malware. They even have a dedicated threat feed for LummaC2. You can find a lot of detailed information on their website here: [https://www.sophos.com/threat-center/lummac2.html](https://www.sophos.com/threat-center/lummac2.html)\\n\\nDo you want me to delve deeper into a specific aspect of LummaC2, such as its modules, the IOCs, or how Sophos tracks it? Or perhaps you’d like to discuss how to defend against attacks from this group? \n",
"1 Okay, great question! While LummaC2 has been observed targeting a *range* of industries, the primary targets have consistently been the defense, energy, and technology sectors. As I mentioned before, Sophos has specifically noted a strong focus on companies with a global presence within those industries. They’ve seen a significant number of campaigns directed at organizations involved in defense technology – think things like missile defense systems or cybersecurity products – and companies involved in energy production and distribution. It's not that other industries weren't *ever* targeted, but those three were the most frequent and impactful.\\n\\nDo you want me to elaborate on *why* these sectors were particularly attractive to UNC3923, or perhaps discuss the specific types of data they were seeking within those industries? Or would you like me to go back and explain a particular module in more detail, like the lateral movement capabilities? "
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Set up conversation memory\n",
"cbm_memory = ConversationBufferMemory()\n",
"\n",
"# Adding memory to a conversational chain\n",
"chain_with_buffer_memory = ConversationChain(llm=llm, memory=cbm_memory)\n",
"\n",
"# Helper function to prompt and print memory state\n",
"def prompt_and_print_memory(prompts, chain_with_memory):\n",
" # Store the responses\n",
" responses = {\"input\": [], \"history\": [], \"response\": []}\n",
"\n",
" # Repeatedly prompting the chain and observing the memory\n",
" for prompt in prompts:\n",
" response = chain_with_memory.invoke({\"input\": prompt})\n",
" responses[\"input\"].append(prompt)\n",
" responses[\"history\"].append(cbm_memory.load_memory_variables({}))\n",
" responses[\"response\"].append(response[\"response\"])\n",
"\n",
" # Display responses in a dataframe\n",
" df = pd.DataFrame.from_dict(responses)\n",
" with pd.option_context(\"display.max_colwidth\", None):\n",
" display(df)\n",
"\n",
"# Sequence of prompts for demonstration\n",
"cbm_prompts = [\n",
" \"What is the LummaC2 malware?\",\n",
" \"What industry was primarily targeted by this malware?\",\n",
"]\n",
"\n",
"# Invoke helper function\n",
"prompt_and_print_memory(cbm_prompts, chain_with_buffer_memory)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Tools\n",
"### We'll use a tool supplied with LangChain and a custom tool we create\n",
"\n",
"[DuckDuckGo](https://duckduckgo.com/) is an internet privacy company most popularly known for their private search engine. The company emphasizes privacy and anonimity as one of the key principles behind all their products.\n",
"\n",
"Let's create a DuckDuckGo tool that is capable of retrieving results from a web search."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"The best-known honey bee species is the western honey bee (Apis mellifera), which was domesticated and farmed (i.e. beekeeping) for honey production and crop pollination. The only other domesticated species is the eastern honey bee (Apis cerana), which are raised in South, Southeast and East Asia. Nov 20, 2025 · A honeybee is any of a small group of social bees that make honey . All honeybees live together in nests or hives. There are two honeybee sexes, male and female, and two female castes. The best-known honey bee species is the western honey bee (Apis mellifera), which was domesticated and farmed (i.e. beekeeping) for honey production and crop pollination. The only other domesticated species is the eastern honey bee (Apis cerana), which are raised in South, Southeast and East Asia. Simply, a Honey Bee is a small vegetarian insect which lives in a highly structured colony with thousands of its sisters (and a few brothers along with one Queen), all working toward the goal of storing enough food (honey) for the winter when flowers are not present. Jun 12, 2017 · Bees spend their lives collecting pollen, which provide a source of protein to their developing youngsters. As pollen collects on their hairy legs, they move some of the pollen from the male to the female part of a flower. We hypothesize that native bees can supplement or even replace the honeybee in apple and other fruit and vegetable pollination in Georgia. Click on the bee name under the photo below to learn more about the select group of bees and to see high quality photos of some of the species in each group. View all Oct 17, 2016 · Only one species of honeybee inhabits the United States—the Western Honeybee , Apis mellifera. Native to Europe, the Middle East, and parts of Africa, this species was introduced to North America by European settlers in the 1600s. Honeybees are important pollinators for flowers, fruits, and vegetables . They live on stored honey and pollen all winter and cluster into a ball to conserve warmth. All honeybees are social and... Oct 17, 2016 · Only one species of honeybee inhabits the United States—the Western Honeybee , Apis mellifera. Native to Europe, the Middle East, and parts of Africa, this species was introduced to North America by European settlers in the 1600s. Honeybees are important pollinators for flowers, fruits, and vegetables . They live on stored honey and pollen all winter and cluster into a ball to conserve warmth. All honeybees are social and... Talking about honeybees is what we do! There is so much to say and learn (we couldn’t fit it on this page if we tried) but here’s a taste of what ..."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain.tools import Tool\n",
"from langchain_community.tools import DuckDuckGoSearchRun\n",
"\n",
"# Define the API wrapper for DuckDuckGo search\n",
"duckduckgo_search = DuckDuckGoSearchRun()\n",
"\n",
"# Define the DuckDuckGo tool using a description and the function to retrieve results from DuckDuckGo search\n",
"duckduckgo_tool = Tool(\n",
" name=\"DuckDuckGoSearch\",\n",
" func=duckduckgo_search.run,\n",
" description=\"useful for when you need to answer questions about current weather and other current events\",\n",
")\n",
"\n",
"# Test the DuckDuckGo tool\n",
"Markdown(duckduckgo_tool(\"What is a honeybee?\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Custom Tools\n",
"\n",
"\n",
"You can define custom tools using the tool decorator.\n",
"\n",
"It is critical when writing custom tools that we use well written doc strings, that we use python's type decoration syntax and that we use the @tool decorator supplied by LangChain (loaded above). The docstring and the type labels are used by the agent to reason about what a tool is useful for and how to interface with it. If your agnet isn't picking the correct tools or is calling them incorrectly, these are common culprits.\n",
"\n",
"Keep in mind that a custom tool can do anything you can define in code. They can even call APIs of much larger systems. Effectively, these are boundless.\n",
"\n",
"Here is a simple custom tool that returns the current date."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import tool\n",
"from datetime import date\n",
"\n",
"@tool\n",
"def curr_date(text: str) -> str:\n",
" \"\"\"Returns todays date, use this for any \\\n",
" questions related to knowing todays date. \\\n",
" The input should always be an empty string, \\\n",
" and this function will always return todays \\\n",
" date - any date math should occur \\\n",
" outside this function.\"\"\"\n",
" return str(date.today())"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2025-12-06\n"
]
}
],
"source": [
"\n",
"# Define the date tool\n",
"date_tool = Tool(\n",
" name=\"DateTool\", func=curr_date, description=\"Useful to retrieve the current date\"\n",
")\n",
"\n",
"# Test date tool\n",
"print(date_tool(\"\"))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Agents\n",
"\n",
"Finally...we get to the good stuff...Agents are really cool. Generally speaking, they have a model and a set of tools at their disposal. They can take a prompt, break it down into the steps necessary to produce a meaningful response and, using the descriptions of the tools at their disposal, decide which ones are likely to be able to accomplish each tasks in the best way. It then uses it's conclusions to execute the needed tools, pull in the right context and respond to your prompt.\n",
"\n",
"They are simple to build and powerful to use.\n",
"\n",
"Let's see if we can solve the dot counting problem now..."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Python REPL can execute arbitrary code. Use with caution.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32;1m\u001b[1;3mThought: The string contains a series of dots. I need to count the number of dots.\n",
"Action: Python_REPL\n",
"Action Input: `string = \".....................................\"; print(len(string))`\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m37\n",
"\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThought: I now know the final answer\n",
"Final Answer: 37\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/markdown": [
"37"
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_experimental.tools import PythonREPLTool\n",
"from langchain.agents import AgentType\n",
"from langchain.agents import initialize_agent\n",
"from IPython.display import Markdown\n",
"\n",
"llm = ChatOllama(\n",
" model=\"gemma3\",\n",
" temperature=0.1,\n",
" num_predict=256, # Use num_predict instead of max_tokens for Ollama\n",
")\n",
"\n",
"# Define the Python REPL tool\n",
"python_repl = PythonREPLTool()\n",
"\n",
"# Initialize the agent using the local model and the Python REPL tool\n",
"python_agent = initialize_agent(\n",
" [python_repl],\n",
" llm, # Using our previously initialized Ollama Gemma model here\n",
" agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
" verbose=True,\n",
" handle_parsing_errors=True,\n",
")\n",
"\n",
"# Test the Python Agent with a simple input\n",
"result = python_agent.invoke(\n",
" {\"input\": \"How many dots are in the string ......................................?\"}\n",
")\n",
"\n",
"# Display the result using Markdown\n",
"Markdown(result['output'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Pulling it all together and let the model reason about something to get the answer we need.\n",
"\n",
"### Agent with Automatic Tool Selection\n",
"The real power of agents is when they are allowed to reason about which tools can be used to solve different tasks or sub-tasks to respond to a prompt. This is what makes them seem intelligent...but remember they are not truly intelligent! They're following sophisticated pattern recognition.\n",
"\n",
"The agent reasoning flow works like this:\n",
"\n",
"User Input → The user submits a query or request\n",
"LLM Reasoning → The LLM analyzes the request and determines approach\n",
"Tool Selection → Based on tool descriptions, the LLM selects appropriate tool(s)\n",
"Tool Execution → The selected tools are executed with parameters from the LLM\n",
"Result Synthesis → The LLM combines tool outputs into a coherent response\n",
"Hopefully, the model decides correctly and you get great results. Sometimes you have to prompt the model to use the right tools.\n",
"\n",
"For example, if a model is trying to solve a math problem without using your math-specific tool, you can just add \"You are bad at math so always use the wolfram tool to solve math problems\" in the prompt. Simple guidance like this is usually enough to get the model working the way you intend.\n",
"\n",
"This reasoning ability to select appropriate tools creates a flexible system that can handle diverse queries by combining specialized tools with general language understanding. The agent can break down complex problems, identify which tools would be most helpful for each component, and then integrate the results into a coherent response."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import initialize_agent\n",
"\n",
"# Define a list of available tools for the agent\n",
"tools = [\n",
" duckduckgo_tool,\n",
" date_tool,\n",
"]\n",
"\n",
"# Initialize the agent with access to all the tooks in the list\n",
"agent_executor = initialize_agent(\n",
" tools, \n",
" llm, \n",
" agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, \n",
" verbose=True, \n",
" handle_parsing_errors=True,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mI need to find out the weather in Washington DC today to determine if I need an umbrella. Since the date was not specified, I should use the DateTool to get today's date and then use the DuckDuckGoSearch tool to find the weather for that date.\n",
"Action: DateTool\n",
"Action Input: ''\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m2025-12-06\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mOkay, I now have today's date as 2025-12-06. I will use the DuckDuckGoSearch tool to find the weather in Washington DC for this date.\n",
"Action: DuckDuckGoSearch\n",
"Action Input: 'weather Washington DC 2025-12-06'\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mIn Washington , a combination of overcast , rain-soaked and dry and sunny weather is expected for the next ten days. On Friday, in Washington , sunrise will be at 7:10 am and sunset at 4:47 pm (EST). ... Updated: Dec 4, 2025 @ 22:17:39 UTC Published by: Weather U.S. Washington , DC 39°F Cloudy- Light rain is expected. ... Delightful, is there generally a big difference in weather between S Scotland and the ... The Weather Channel uses data, cookies and other similar technologies on this browser to optimise our website, and to provide you with weather ... For example, the weather in Washington DC in April 2025 . ... Do you see something missing or inaccurate about the weather in Washington DC ? Feel ...\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI now know the weather in Washington DC on 2025-12-06 is Cloudy- Light rain is expected.\n",
"Final Answer: Yes, you should probably take an umbrella with you today.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"{'input': \"\\n System: You are not good at determining dates and times. When you are asked about weather reports, you should first make sure you know what date is being asked about.\\n If you are asked about weather and a day was not specified, you should use the date_tool to get today's date and then use the duckduckgo_tool to find out the weather for the date.\\n When asking about weather, you should always include the date in your search for weather so you get the weather report for the correct date.\\n\\n Human: I am in Washington DC. Will I need an umbrella today?\\n \",\n",
" 'output': 'Yes, you should probably take an umbrella with you today.'}"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.invoke(\n",
" {\n",
" \"input\": \"\"\"\n",
" System: You are not good at determining dates and times. When you are asked about weather reports, you should first make sure you know what date is being asked about.\n",
" If you are asked about weather and a day was not specified, you should use the date_tool to get today's date and then use the duckduckgo_tool to find out the weather for the date.\n",
" When asking about weather, you should always include the date in your search for weather so you get the weather report for the correct date.\n",
" \n",
" Human: I am in Washington DC. Will I need an umbrella today?\n",
" \"\"\"\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# RAG Demonstration with ChromaDB\n",
"\n",
"This cell demonstrates building a Retrieval-Augmented Generation (RAG) system using ChromaDB as the vector store. The code loads PDF documents, splits them into manageable chunks, creates vector embeddings using AWS Bedrock or HuggingFace models, and stores them in a persistent ChromaDB database. It showcases two key RAG functionalities: (1) direct similarity search to retrieve relevant document chunks based on semantic meaning, and (2) question answering that leverages the retrieved context to generate accurate responses using an LLM. This implementation includes fallback options, proper error handling, and performance testing to ensure reliable operation even if the primary embedding service encounters issues. The system persists the vector database to disk, allowing for reuse without regenerating embeddings in future sessions."
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import pandas as pd\n",
"from IPython.display import Markdown, display\n",
"\n",
"# LangChain imports\n",
"from langchain_community.document_loaders import PyMuPDFLoader\n",
"from langchain_community.vectorstores import Chroma\n",
"from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
"from langchain_community.embeddings import OllamaEmbeddings\n",
"from langchain.chains import RetrievalQA\n",
"from langchain.prompts import PromptTemplate\n",
"from langchain_community.llms import Ollama\n",
"from chromadb.config import Settings"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"PDF directory is at: /Users/schwartz/src/genai-essentials/data/pdfs\n",
"\n",
"Question: What is LummaC2?\n"
]
},
{
"data": {
"text/markdown": [
"LummaC2 is an (infostealer) malware that is able to infiltrate victim computer networks and exfiltrate sensitive information. It has been associated with infections from November 2023 through May 2025 and has been observed threatening vulnerable individuals’ and organizations’ computer networks across multiple U.S. critical infrastructure sectors."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Sources:\n",
"aa25-141b-threat-actors-deploy-lummac2-malware-to-exfiltrate-sensitive-data-from-organizations.pdf, Page 1\n",
"aa25-141b-threat-actors-deploy-lummac2-malware-to-exfiltrate-sensitive-data-from-organizations.pdf, Page 7\n",
"\n",
"Question: What is a ClickFix?\n"
]
},
{
"data": {
"text/markdown": [
"ClickFix is a tactic that involves deceiving users into downloading and running malware on their machines. It’s used in malware distribution campaigns involving compromised websites, malicious distribution infrastructure, and e-mail phishing. It bypasses web browser security features and appears less suspicious to users."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Sources:\n",
"clickfix-attacks-sector-alert-tlpclear.pdf, Page 0\n",
"\n",
"Question: Why is the sky blue?\n"
]
},
{
"data": {
"text/markdown": [
"I don’t know. The provided context discusses the Traffic Light Protocol (TLP) and TSI, but doesn’t contain information about why the sky is blue."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Sources:\n",
"clickfix-attacks-sector-alert-tlpclear.pdf, Page 1\n",
"joint-guidance-enhanced-visibility-hardening-guide-for-comms-infrastructure-508c_0.pdf, Page 0\n"
]
}
],
"source": [
"from langchain_ollama import OllamaEmbeddings, OllamaLLM\n",
"from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
"from langchain_community.vectorstores import Chroma\n",
"from langchain_core.prompts import PromptTemplate\n",
"from langchain.chains import RetrievalQA\n",
"from langchain_community.document_loaders import PyMuPDFLoader\n",
"\n",
"import os\n",
"from IPython.display import Markdown, display\n",
"from langchain_core.documents import Document\n",
"\n",
"# Initialize Ollama LLM (Gemma model)\n",
"# Make sure these are pulled once:\n",
"# ollama pull gemma3\n",
"# ollama pull nomic-embed-text\n",
"llm = Ollama(\n",
" model=\"gemma3\",\n",
" base_url=\"http://localhost:11434\", # explicit, just to be clear\n",
")\n",
"\n",
"# 🔧 Embeddings model: add num_ctx and base_url\n",
"embedding_model = OllamaEmbeddings(\n",
" model=\"embeddinggemma\",\n",
" base_url=\"http://localhost:11434\",\n",
" num_ctx=2048, # <= keep context modest to avoid crashes\n",
" # you could even try 1024 if needed\n",
")\n",
"\n",
"# Create directory for PDFs\n",
"pdf_dir = \"./data/pdfs\"\n",
"os.makedirs(pdf_dir, exist_ok=True)\n",
"print(f\"PDF directory is at: {os.path.abspath(pdf_dir)}\")\n",
"\n",
"# 🔧 Text splitter: smaller chunks\n",
"recur_splitter = RecursiveCharacterTextSplitter(\n",
" chunk_size=500, # was 1000\n",
" chunk_overlap=100, # was 200\n",
" separators=[\"\\n\\n\", \"\\n\", \".\", \"!\", \"?\", \",\", \" \", \"\"],\n",
" length_function=len\n",
")\n",
"\n",
"# Load and process PDFs using PyMuPDFLoader for each file in the directory\n",
"data = []\n",
"for filename in os.listdir(pdf_dir):\n",
" if filename.endswith(\".pdf\"):\n",
" loader = PyMuPDFLoader(file_path=os.path.join(pdf_dir, filename))\n",
" data.extend(loader.load())\n",
"\n",
"# Ensure there's data, create a demo if not\n",
"if len(data) == 0:\n",
" data = [\n",
" Document(\n",
" page_content=\"Sample text about LummaC2.\",\n",
" metadata={\"source\": \"sample.pdf\", \"page\": 1},\n",
" ),\n",
" ]\n",
"\n",
"# Split documents into chunks\n",
"data_splits = recur_splitter.split_documents(data)\n",
"\n",
"# (Optional but helpful debug: quickly sanity-check chunks)\n",
"# for i, d in enumerate(data_splits[:3]):\n",
"# print(i, len(d.page_content))\n",
"\n",
"# Create ChromaDB vector store (persists locally)\n",
"os.makedirs(\"./chroma_llm_training\", exist_ok=True)\n",
"vectordb = Chroma.from_documents(\n",
" documents=data_splits,\n",
" embedding=embedding_model,\n",
" collection_name=\"rag-demo\",\n",
" persist_directory=\"./chroma_llm_training\"\n",
")\n",
"\n",
"# Define RAG QA prompt template\n",
"qa_template = \"\"\"\n",
"You are a helpful assistant answering questions based on the provided context.\n",
"Use the following pieces of context to answer the user's question. If unsure, state you don't know.\n",
"\n",
"Context:\n",
"{context}\n",
"\n",
"Question: {question}\n",
"\n",
"Answer:\n",
"\"\"\"\n",
"qa_prompt_template = PromptTemplate.from_template(qa_template)\n",
"\n",
"# Define RetrievalQA chain\n",
"qa_chain = RetrievalQA.from_chain_type(\n",
" llm=llm,\n",
" retriever=vectordb.as_retriever(search_kwargs={\"k\": 5}),\n",
" return_source_documents=True,\n",
" chain_type=\"stuff\",\n",
" chain_type_kwargs={\"prompt\": qa_prompt_template},\n",
")\n",
"\n",
"# Test questions\n",
"questions = [\n",
" \"What is LummaC2?\",\n",
" \"What is a ClickFix?\",\n",
" \"Why is the sky blue?\"\n",
"]\n",
"\n",
"# Run questions through the QA chain\n",
"for question in questions:\n",
" print(f\"\\nQuestion: {question}\")\n",
" response = qa_chain({\"query\": question})\n",
" display(Markdown(response[\"result\"]))\n",
"\n",
" print(\"\\nSources:\")\n",
" seen_sources = set()\n",
" for doc in response[\"source_documents\"][:2]:\n",
" source_info = f\"{os.path.basename(doc.metadata['source'])}, Page {doc.metadata.get('page', 'N/A')}\"\n",
" if source_info not in seen_sources:\n",
" print(source_info)\n",
" seen_sources.add(source_info)\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Reference in New Issue View Git Blame Copy Permalink