admin/trailofbits-buttercup
1
0
Fork 0
mirror of https://github.com/trailofbits/buttercup synced 2026-06-21 14:11:39 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: Add local SigNoz deployment support (#232)

Browse Source 
* feat: Add simplified SigNoz integration using official Helm chart

- Add SigNoz as a Helm dependency with conditional deployment
- Enable SigNoz by default for minikube environments
- Auto-configure OTEL to use internal SigNoz when enabled
- Update Docker Compose to include existing SigNoz stack
- Minimal configuration following the official chart approach

Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>

* Configure local signoz

* do not deploy signoz in ci

* makefile: ensure signoz is available

* feat: Simplify SigNoz integration for improved user experience

- Remove external SigNoz configuration prompts from setup-local script
- Make local SigNoz deployment the default for quickstart experience
- Move external SigNoz configuration to MANUAL_SETUP.md for advanced users
- Streamline README log access section to focus on local SigNoz UI
- Relocate kubectl commands to QUICK_REFERENCE.md as alternative method
- Improve documentation structure for better user onboarding

Addresses feedback from @michaelbrownuc to simplify the integration
before merging.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>

* Update MANUAL_SETUP.md

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Riccardo Schirone <ret2libc@users.noreply.github.com>
This commit is contained in:
Riccardo Schirone
2025-08-06 15:35:53 +02:00
committed by GitHub
parent cfd0441584
commit a8ed0ccbbe
19 changed files with 140 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/ci-env.template
+1
View File
@@ -1,5 +1,6 @@
export BUTTERCUP_K8S_VALUES_TEMPLATE=./k8s/values-upstream-minikube.template
export DEPLOY_CLUSTER=true
export DEPLOY_SIGNOZ=false
export CLUSTER_TYPE=minikube
export TAILSCALE_ENABLED=false
export COMPETITION_API_ENABLED=false
MANUAL_SETUP.md
+18
View File
@@ -69,6 +69,24 @@ cp deployment/env.template deployment/env
Look at the comments in the `deployment/env.template` for how to set variables.
### External SigNoz Configuration
If you want to use an external SigNoz instance instead of the default local deployment, you can configure custom OpenTelemetry settings:
1. **Disable local SigNoz deployment:**
```bash
# In deployment/env
export DEPLOY_SIGNOZ=false
```
2. **Configure external OTEL endpoint:**
```bash
# In deployment/env
export OTEL_ENDPOINT="https://your-signoz-instance.com"
export OTEL_PROTOCOL="http" # or "grpc"
export OTEL_TOKEN="your-otel-token" # optional
```
## Start Services Manually
```bash
Makefile
+21
View File
@@ -20,6 +20,7 @@ help:
@echo " status - Check the status of the deployment"
@echo " crs-instance-id - Get the CRS instance ID"
@echo " download-artifacts - Download submitted artifacts from the CRS"
@echo " signoz-ui - Open the SigNoz UI"
@echo ""
@echo "Testing:"
@echo " send-integration-task - Run integration-test task"
@@ -213,6 +214,26 @@ clean-local:
install-cscope:
cd external/aixcc-cscope/ && autoreconf -i -s && ./configure && make && sudo make install
signoz-ui:
@echo "Opening SigNoz UI..."
@if ! kubectl get namespace $${BUTTERCUP_NAMESPACE:-crs} >/dev/null 2>&1; then \
echo "Error: CRS namespace not found. Deploy first with 'make deploy'."; \
exit 1; \
fi
@if ! kubectl get service/buttercup-signoz-frontend -n $${BUTTERCUP_NAMESPACE:-crs} >/dev/null 2>&1; then \
echo "Error: SigNoz is not deployed. Set DEPLOY_SIGNOZ=true in deployment/env and redeploy."; \
exit 1; \
fi
kubectl port-forward -n $${BUTTERCUP_NAMESPACE:-crs} service/buttercup-signoz-frontend 33301:3301 &
@sleep 3
@if command -v xdg-open >/dev/null 2>&1; then \
xdg-open http://localhost:33301; \
elif command -v open >/dev/null 2>&1; then \
open http://localhost:33301; \
else \
echo "Please open http://localhost:33301 in your browser."; \
fi
web-ui:
@echo "Opening web UI (https://localhost:31323/)..."
@if ! kubectl get namespace $${BUTTERCUP_NAMESPACE:-crs} >/dev/null 2>&1; then \
QUICK_REFERENCE.md
+3 -1
View File
@@ -40,8 +40,10 @@ kubectl get services -n crs
# Port forwarding
kubectl port-forward -n crs service/buttercup-competition-api 31323:1323
# View logs
# View logs (alternative to SigNoz UI)
kubectl get pods -n crs
kubectl logs -n crs <pod-name>
kubectl logs -n crs <pod-name> -f # Follow logs in real-time
kubectl logs -n crs -l app=scheduler --tail=-1 --prefix
# Debug pods
README.md
+16 -14
View File
@@ -22,6 +22,7 @@
**Note:** Buttercup works best with access to models from OpenAI **and** Anthropic, but can be run with at least one API key from one third-party provider (support for Gemini coming soon).
### Supported Systems
- **Linux x86_64** (fully supported)
- **ARM64** (partial support for upstream Google OSS-Fuzz projects)
@@ -34,7 +35,7 @@ Before setup, ensure you have these packages installed:
sudo apt-get update
sudo apt-get install -y make curl git
# RHEL/CentOS/Fedora
# RHEL/CentOS/Fedora
sudo yum install -y make curl git
# or
sudo dnf install -y make curl git
@@ -84,12 +85,11 @@ make status
When a deployment is successful, you should see all pods in "Running" or "Completed" status.
5. Send Buttercup a simple task
**Note:** When tasked, Buttercup will start consuming third-party AI resources.
**Note:** When tasked, Buttercup will start consuming third-party AI resources.
This command will make Buttercup pull down an example repo [example-libpng](https://github.com/tob-challenges/example-libpng) with a known vulnerability. Buttercup will start fuzzing it to find and patch vulnerabilities.
This command will make Buttercup pull down an example repo [example-libpng](https://github.com/tob-challenges/example-libpng) with a known vulnerability. Buttercup will start fuzzing it to find and patch vulnerabilities.
```bash
make send-libpng-task
@@ -107,7 +107,6 @@ Then navigate to `http://localhost:31323` in your web browser.
In the GUI you can monitor active tasks and see when Buttercup finds bugs and generates patches for them.
7. Stop Buttercup
**Note:** This is an important step to ensure Buttercup shuts down and stops consuming third-party AI resources.
@@ -118,19 +117,22 @@ make undeploy
## Accessing Logs
To view system logs and telemetry you can use Langfuse or SigNoz if you configured them during setup. Otherwise you can access logs via `kubectl` commands:
Buttercup includes local SigNoz deployment by default for comprehensive system observability. You can access logs, traces, and metrics through the SigNoz UI:
```bash
# View all pods
kubectl get pods -n crs
# View specific pod logs
kubectl logs -n crs <pod-name>
# Follow logs in real-time
kubectl logs -n crs <pod-name> -f
make signoz-ui
```
Then navigate to `http://localhost:33301` in your web browser to view:
- Distributed traces
- Application metrics
- Error monitoring
- Performance insights
If you configured LangFuse during setup, you can also monitor LLM usage and costs there.
For additional log access methods, see the [Quick Reference Guide](QUICK_REFERENCE.md).
## Additional Resources
- [Quick Reference Guide](QUICK_REFERENCE.md) - Common commands and troubleshooting
deployment/crs-architecture.sh
+1
View File
@@ -37,6 +37,7 @@ fi
BUTTERCUP_NAMESPACE=${BUTTERCUP_NAMESPACE:-crs}
DEPLOY_CLUSTER=${DEPLOY_CLUSTER:-true}
DEPLOY_SIGNOZ=${DEPLOY_SIGNOZ:-false}
CLUSTER_TYPE=${CLUSTER_TYPE:-minikube}
if [ "$DEPLOY_CLUSTER" = "true" ] && [ "$CLUSTER_TYPE" = "aks" ]; then
deployment/env.template
+2 -1
View File
@@ -72,8 +72,9 @@ export ANTHROPIC_API_KEY="<your-anthropic-api-key>"
# export LANGFUSE_SECRET_KEY="<your-langfuse-secret-key>"
# OpenTelemetry endpoint settings (e.g. SigNoz)
export DEPLOY_SIGNOZ=false
# export OTEL_ENDPOINT="<your-otel-endpoint>"
# export OTEL_TOKEN="<your-otel-http-token>"
# export OTEL_TOKEN="dXNlcm5hbWU6cGFzc3dvcmQ="
# export OTEL_PROTOCOL=grpc
# Docker Hub credentials for doing logged requests while getting Container
deployment/k8s/.gitignore
+5
View File
@@ -1,3 +1,8 @@
charts/dind-daemon-*.tgz
charts/pov-reproducer-*.tgz
charts/registry-cache-*.tgz
charts/scratch-cleaner-*.tgz
charts/ui-*.tgz
charts/build-bot-*.tgz
charts/competition-api-*.tgz
charts/coverage-bot-*.tgz
deployment/k8s/Chart.lock
+20 -2
View File
@@ -2,6 +2,9 @@ dependencies:
- name: redis
repository: https://charts.bitnami.com/bitnami
version: 17.17.1
- name: dind-daemon
repository: file://charts/dind-daemon
version: 0.1.0
- name: litellm-helm
repository: oci://ghcr.io/berriai
version: 0.1.615
@@ -38,5 +41,20 @@ dependencies:
- name: competition-api
repository: file://charts/competition-api
version: 0.0.1
digest: sha256:8ac9c249917d32604f6d59d65bc74e7a7a7eb46b7c71fbb982bd7b0dd038395a
generated: "2025-03-26T13:45:05.979619483Z"
- name: registry-cache
repository: file://charts/registry-cache
version: 0.1.0
- name: pov-reproducer
repository: file://charts/pov-reproducer
version: 0.1.0
- name: scratch-cleaner
repository: file://charts/scratch-cleaner
version: 0.0.1
- name: ui
repository: file://charts/ui
version: 0.1.0
- name: signoz
repository: https://charts.signoz.io
version: 0.71.0
digest: sha256:7175e733201e2be1f90820b1515a650edc4afefe4c57cc6fb9978a03bbdf6920
generated: "2025-08-04T14:19:11.272734+02:00"
deployment/k8s/Chart.yaml
+5 -1
View File
@@ -56,5 +56,9 @@ dependencies:
version: "0.0.1"
repository: "file://charts/scratch-cleaner"
- name: ui
version: "0.0.1"
version: "0.1.0"
repository: "file://charts/ui"
- name: signoz
version: "0.71.0"
repository: "https://charts.signoz.io"
condition: signoz.enabled
deployment/k8s/charts/signoz-0.71.0.tgz
BIN
View File
Binary file not shown.
deployment/k8s/templates/common-env.yaml
+9
View File
@@ -85,12 +85,21 @@ Define Docker Host environment variable for Unix socket
{{- end }}
{{- define "buttercup.env.telemetry" }}
{{- if .Values.global.signoz.deployed }}
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "http://{{ .Release.Name }}-signoz-otel-collector:4317"
- name: OTEL_EXPORTER_OTLP_HEADERS
value: "Authorization=Basic dXNlcm5hbWU6cGFzc3dvcmQ="
- name: OTEL_EXPORTER_OTLP_PROTOCOL
value: "grpc"
{{- else }}
- name: OTEL_EXPORTER_OTLP_ENDPOINT
value: "{{ .Values.global.otel.endpoint }}"
- name: OTEL_EXPORTER_OTLP_HEADERS
value: "Authorization=Basic {{ .Values.global.otel.token }}"
- name: OTEL_EXPORTER_OTLP_PROTOCOL
value: "{{ .Values.global.otel.protocol | default "grpc" }}"
{{- end }}
- name: CRS_INSTANCE_ID
valueFrom:
configMapKeyRef:
deployment/k8s/values-upstream-minikube.template
+6
View File
@@ -27,6 +27,9 @@ global:
publicKey: "${LANGFUSE_PUBLIC_KEY}"
secretKey: "${LANGFUSE_SECRET_KEY}"
signoz:
deployed: ${DEPLOY_SIGNOZ}
crs:
api_key_id: ${CRS_KEY_ID}
api_key_token: ${CRS_KEY_TOKEN}
@@ -81,3 +84,6 @@ image-preloader:
- gcr.io/oss-fuzz-base/base-builder-rust
versions:
- latest
signoz:
enabled: ${DEPLOY_SIGNOZ}
deployment/k8s/values.yaml
+5
View File
@@ -47,6 +47,8 @@ global:
host: "https://cloud.langfuse.com"
publicKey: "pk-lf-..." # Replace with your actual public key
secretKey: "sk-lf-..." # Replace with your actual secret key
signoz:
deployed: false
crs:
api_key_id: 515cc8a0-3019-4c9f-8c1c-72d0b54ae561
api_key_token: VGuAC8axfOnFXKBB7irpNDOKcDjOlnyB
@@ -375,3 +377,6 @@ litellm-helm:
restartPolicy: Always
# SigNoz observability platform
signoz:
enabled: false
dev/docker-compose/compose.yaml
+1
View File
@@ -1,5 +1,6 @@
include:
- ../../competition-server/compose.yaml
- ../../competition-server/signoz/compose.yaml
services:
# ugh the only solution i can come up with is a dind instance paired with every single build-bot we will need like k8s groups or something
dev/docker-compose/env.dev.compose
+7 -8
View File
@@ -73,16 +73,15 @@ INDEX_TASK_TIMEOUT_MS=120000
INDEX_OUTPUT_TASK_TIMEOUT_MS=120000
TRACED_VULNERABILITIES_TASK_TIMEOUT_MS=120000
# OTel
# Settings for locally hosted endpoint, currently disabled.
#OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
# OTel - configured to use local SigNoz by default
OTEL_EXPORTER_OTLP_ENDPOINT=http://signoz-otel-collector:4317
# token is b64 encoded "username:password"
#OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic dXNlcm5hbWU6cGFzc3dvcmQ="
#OTEL_EXPORTER_OTLP_PROTOCOL="grpc"
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic dXNlcm5hbWU6cGFzc3dvcmQ="
OTEL_EXPORTER_OTLP_PROTOCOL="grpc"
# Settings for externally hosted endpoint, make sure to set endpoint to url (kept secret)
OTEL_EXPORTER_OTLP_ENDPOINT=""
OTEL_EXPORTER_OTLP_PROTOCOL="http"
# Settings for externally hosted endpoint, uncomment and set endpoint to use external OTEL
#OTEL_EXPORTER_OTLP_ENDPOINT=""
#OTEL_EXPORTER_OTLP_PROTOCOL="http"
OSS_FUZZ_CONTAINER_ORG=aixcc-afc
scripts/common.sh
+18 -7
View File
@@ -544,20 +544,29 @@ configure_llm_budget() {
}
# Function to configure OTEL telemetry
# Function to configure OTEL telemetry (simplified for local deployment)
configure_otel() {
print_linebreak
print_status "Configuring OpenTelemetry telemetry (optional)..."
print_status "Configuring SigNoz for local observability..."
# Source the env file to check current values
if [ -f "deployment/env" ]; then
source deployment/env
fi
print_status "OpenTelemetry: Optional distributed tracing and metrics collection."
print_status "SigNoz: Local observability platform for distributed tracing and metrics."
print_status "Provides detailed performance monitoring and system observability for debugging."
configure_service "OTEL" "OpenTelemetry configuration" "$OTEL_ENDPOINT" "<your-otel-endpoint>" false "configure_otel_wrapper"
# Check if already configured and user wants to keep it
if [ "$DEPLOY_SIGNOZ" = "true" ]; then
if ! prompt_for_update "DEPLOY_SIGNOZ" "SigNoz deployment" "local" "" false; then
return 0 # User chose to keep existing value, exit early
fi
fi
# Enable local SigNoz deployment by default for quickstart
portable_sed "s|.*export DEPLOY_SIGNOZ=.*|export DEPLOY_SIGNOZ=true|" deployment/env
print_success "Local SigNoz deployment enabled for observability"
}
# Function to check configuration file
@@ -662,8 +671,10 @@ check_aks_config() {
done
fi
# Check optional OTEL configuration
if [ -n "$OTEL_ENDPOINT" ] && [ "$OTEL_ENDPOINT" != "" ]; then
# Check optional SigNoz/OTEL configuration
if [ "$DEPLOY_SIGNOZ" = "true" ]; then
print_status "SigNoz local deployment is enabled"
elif [ -n "$OTEL_ENDPOINT" ] && [ "$OTEL_ENDPOINT" != "" ]; then
if [ -z "$OTEL_PROTOCOL" ] || [ "$OTEL_PROTOCOL" = "<your-*>" ]; then
print_error "OTEL_PROTOCOL is not set when OTEL_ENDPOINT is configured"
errors=$((errors + 1))
scripts/setup-azure.sh
+1 -1
View File
@@ -79,7 +79,7 @@ setup_config() {
# Configure LangFuse (optional)
configure_langfuse
# Configure OTEL telemetry (optional)
# Configure SigNoz deployment (optional)
configure_otel
}
scripts/setup-local.sh
+1 -1
View File
@@ -27,7 +27,7 @@ setup_config() {
# Configure LangFuse (optional)
configure_langfuse
# Configure OTEL telemetry (optional)
# Configure SigNoz deployment (optional)
configure_otel
}