Commit Graph

27 Commits

Author SHA1 Message Date
Will Schroeder a36be30b39 K8s k3d/k3s support (#106)
* Add k8s/k3d/k3s deployment with Helm, KEDA autoscaling, and connection pool fixes

- Helm chart for full Nemesis deployment (Traefik, Dapr, KEDA, all services)
- KEDA autoscaling for file-enrichment and document-conversion on RabbitMQ queue depth
- Fix PostgreSQL connection exhaustion: configurable pool size (default 20, was 100)
- Cap Dapr statestore connections (maxConns=10), increase postgres max_connections to 300
- Add Prometheus metrics and /system/pool-stats endpoint for connection pool monitoring
- Background pool stats logger with pressure warnings
- Fix Prometheus scrape targets to correct service ports
- Helm-based Dapr install (v1.16.9) instead of CLI
- k8s setup/deploy/build/verify/teardown scripts
- Kubernetes deployment docs and references in CLAUDE.md, AGENTS.md, mkdocs

* Add KEDA autoscaling for titus-scanner and dotnet-service

* Add PgBouncer connection pooling, KEDA gotenberg CPU scaling, and tuning

- Add PgBouncer deployment between services and PostgreSQL (transaction pooling)
- Route all DB connections through pgbouncer instead of postgres directly
- Add KEDA CPU-based autoscaling for gotenberg (1-3 replicas)
- Mirror external images (pgbouncer) into k3d registry via build-and-push.sh
- Reduce queue-based cooldowns to 60s, gotenberg stabilization to 30s
- Add 180s terminationGracePeriodSeconds to gotenberg for in-flight conversions
- Update docs with PgBouncer architecture, connection tuning, and troubleshooting

* Add k3s support, unify Traefik on port 7443, rename scripts

- Add setup-cluster-k3s.sh and teardown-cluster-k3s.sh for native k3s
- Rename scripts with -k3d suffix to distinguish runtimes
- Unify k3s Traefik to HTTPS-only on port 7443 (matching k3d)
- Add k3d registry check for --build flag in deploy.sh
- Fix verify.sh arithmetic under set -e with || true
- Update docs with k3s quick start, comparison table, and teardown
- Add image-sync workflow and pgbouncer registry override for values-dev

* Add monitoring, Jupyter, and LLM optional stacks to Helm chart

- Monitoring: Prometheus, Grafana (with dashboards), Loki, Promtail,
  Jaeger, OTEL Collector, Node Exporter, Postgres Exporter
- Jupyter: notebook service with Hasura integration
- LLM: Agents (Dapr-enabled), LiteLLM proxy, Phoenix observability
- Add ingress routes, strip-prefix middleware, litellm database
- Update deploy.sh, build script, README, and test connectivity checks

* Add k3s --build support with auto-detection in deploy.sh

- deploy.sh --build auto-detects k3d registry vs k3s and uses appropriate build script
- Add build-and-load-k3s.sh for building and importing images into k3s containerd
- Add values-dev-k3s.yaml with imagePullPolicy: Never (no pgbouncer registry override)
- Fix jupyter Dockerfile context path in build-and-push-k3d.sh
- Update docs for k3s local build support

---------

Co-authored-by: Lee Chagolla-Christensen <lee@localhost>
2026-03-05 10:29:03 -08:00
Lee Chagolla-Christensen f2bc83a5fa spelling/grammar in quickstart.md 2025-11-12 13:57:58 -08:00
harmj0y 78390b410b PII analyzer disabled by default, added documentation
- PII analyzer disabled by default, can be enabled via ENV variable
- Created `enrichment_configuration.md` documentation documenting all enrichment module configurations
2025-11-06 11:08:59 -08:00
harmj0y d55076fd8e doc update
doc update
2025-10-27 13:58:01 -07:00
harmj0y 0748f5abbc Added "Text Translator" agent
-Added "Text Translator" agent
2025-10-02 21:05:07 +09:00
Lee Chagolla-Christensen 9b2e8a4ce8 fix indent 2025-07-01 19:30:00 -07:00
harmj0y 545bf502d8 Update CHANGELOG.md
-Update CHANGELOG.md
2025-06-27 13:53:04 -07:00
Lee Chagolla-Christensen 53f3f3ec5b link to env.example in docs 2025-06-27 11:59:58 -07:00
Lee Chagolla-Christensen 9aa13db594 add FQDN recommendation 2025-06-27 11:18:50 -07:00
harmj0y cf29b7fc6f Updated quickstart.md docs
-Updated quickstart.md docs
2025-06-27 09:47:44 -07:00
Lee Chagolla-Christensen 1ce9dde651 doc updates 2025-06-26 14:02:38 -07:00
Lee Chagolla-Christensen b6adcf4efd add advanced docker compose stuff 2025-06-26 13:52:17 -07:00
Lee Chagolla-Christensen a5ccdd0fc1 update quickstart 2025-06-26 12:40:33 -07:00
Lee Chagolla-Christensen 9908a7a992 update start.sh 2025-06-25 22:27:02 -07:00
harmj0y 88ffe1422f Additional documentation
-How to change the Nemesis port
-How to change the SSL certs
-Performance tuning guidance
2025-06-19 15:35:07 +03:00
harmj0y 1ebb27c4e4 Update quickstart.md 2025-06-13 15:33:10 +02:00
Lee Christensen 8035a6a168 add code 2025-06-13 11:33:07 +02:00
Lee Christensen 889613f44c initial 2.0 commit 2025-06-13 11:28:53 +02:00
Will f1665e75ab Update quickstart.md 2024-04-25 16:29:45 -07:00
Will 2e788f19ea Documentation tweaks
Bumped version, in README, added more thanks, added blog posts
2024-04-25 15:01:17 -07:00
Max Harley e54317937c Update quickstart.md 2024-04-24 16:24:33 -04:00
Max Harley 26aef693e0 Update documentation 2024-04-24 13:27:03 -04:00
Max Harley ac1c453bc3 Update quickstart 2024-04-23 16:36:13 -04:00
Max Harley 1e6aee33b2 Update documentation 2024-04-23 12:09:55 -04:00
Max Harley 90ef5c6097 Update documentation 2024-04-23 11:54:42 -04:00
Max Harley 843acac41a Update documentation 2024-04-23 11:52:39 -04:00
Max Harley 0ba7097514 Update documentation 2024-04-23 11:51:24 -04:00