Skip to main content
This guide walks you through deploying the Genesis container to an existing AWS EKS (Elastic Kubernetes Service) cluster.
For everything after the install — upgrade, rollback, backup and restore, secrets, user offboarding, scaling, teardown, and the failure-mode catalogue — see the Genesis Operations Runbook. It is cloud-agnostic, with a per-cloud section for the EKS-specific substrate. EBS snapshot procedures stay in Backup and Recovery.
Genesis will provide you with the container image URL. Replace <container-registry-url> throughout this document with the URL provided by Genesis. Helm charts are published as private OCI artifacts on Docker Hub (oci://registry-1.docker.io/genesiscomputingai/genesis), ECR (oci://<account>.dkr.ecr.<region>.amazonaws.com/charts/genesis), and GHCR (oci://ghcr.io/genesis-computing-ai/charts/genesis). Docker Hub and ECR credentials are provided during onboarding; GHCR access uses a GitHub PAT with read:packages. If you use the bundled aws-eks/ helper scripts, copy aws-eks/eks-config-example.env to aws-eks/eks-config.env, set ECR_REGISTRY, and ensure AWS_DEFAULT_REGION matches the ECR region so aws ecr get-login-password succeeds. Pin image.tag in your values file, and set GENESIS_IMAGE_TAG in aws-eks/eks-config.env only when you want a one-off override for a deploy.

Prerequisites

AWS EKS Cluster Requirements

EKS Cluster

Kubernetes version 1.23 or higher

kubectl

Configured to access your EKS cluster

Helm 3.8+

For deploying the Genesis Helm chart

AWS CLI

Configured with appropriate credentials
Verify cluster access:
Verify Helm installation:

Additional Requirements

Network Access

IAM Permissions

The EKS cluster node group IAM role needs the following permissions:
  • AmazonEKSWorkerNodePolicy
  • AmazonEKS_CNI_Policy
  • AmazonEC2ContainerRegistryReadOnly
If using AWS Bedrock as the LLM provider, use IAM Roles for Service Accounts (IRSA) for production least-privilege access. Granting Bedrock permissions to the node group role can work for simple non-production deployments, but it gives every workload on that node role the same access.

EBS CSI Driver Setup

Genesis requires persistent storage for its database, git repositories, and uploaded files. The Amazon EBS CSI driver must be installed in your cluster to provision EBS volumes.
The EBS CSI driver is not automatically installed by default on standard EKS clusters (even for Kubernetes 1.23+). You must install it manually. However, if you’re using EKS Auto Mode, the EBS CSI driver is automatically included and no installation is needed.
Check if installed:

Installation Instructions

Refer to the Amazon EKS User Guide for detailed EBS CSI driver installation steps, including IAM role setup and add-on installation.

Ingress Controller Setup

Genesis requires an ingress controller for external access. Choose one based on your requirements:
Single-instance and load balancer choice: Genesis runs as a single pod (genesis-0) and does not support multiple instances. ALB/NLB therefore do not add value for load distribution across replicas. They still add value for: exposing the app with a stable external endpoint, TLS termination (e.g. ACM with ALB), health checks, and integration with Route53, WAF, etc. You can also expose Genesis using a LoadBalancer-type Service (one NLB/ELB per service) if you prefer not to use an ingress controller.

Genesis Container Configuration

Container Image

Current Genesis container images include ripgrep (rg) out of the box for in-container file-content searches. After kubectl exec, you can use it to inspect mounted files under /app/bot_storage or /app/bot_git without installing extra packages. The same images also bundle the in-repo genesis_sdk package under /app/genesis_sdk (for execute_python_code local execution — not installable from PyPI) and the llm_wiki package under /app/llm_wiki, so the SDK, wiki APIs, and tools shipped with Genesis are available without extra image customization.

Image Pull Secrets (If Required)

If the container registry requires authentication, Genesis will provide the necessary credentials. Create a Kubernetes secret:
Then reference it in your Helm values file:

Exposed Ports

Persistent Storage Requirements

Genesis requires persistent storage for:
Recommended Storage Size: 50Gi minimum (adjust based on expected usage)Storage Class: Uses cluster default (typically gp2 or gp3 on EKS)
Before enabling Claude Code on EKS, review the worktree and run-evidence persistence notes in Cursor and Claude Code coding backends.

Environment Variables

The active LLM provider belongs to APP_CONFIG. Configure Bedrock after the pod is running with the config API or Config -> AI Providers — see Configuring a provider through the config API and the worked example under Deployment Steps below.The CloudFormation BedrockModelId parameter and Terraform bedrock_model_id input default to "". Leave them empty to use the Bedrock catalog defaults, including Claude Opus 5.5 (global.anthropic.claude-opus-5-5) for chat, or set a model id to pin it. The Terraform AWS and ECS templates seed an unset provider configuration with ACTIVE_LLM_PROVIDER=litellm_bedrock and BEDROCK_AUTH_MODE=iam_role. They also set BEDROCK_ENABLED=true so images through v1.19.95 recognize the provider when no configured marker is stored; newer images ignore this switch. A non-empty model input also seeds BEDROCK_MODEL_NAME, BEDROCK_FAST_MODEL_NAME, and BEDROCK_PREMIER_MODEL. When the input is empty, those templates omit the model overrides. Saved configuration or a mounted genesis-config.yaml takes precedence over these deployment defaults.
These are the standard AWS SDK region variables that boto3 reads; they do not select or configure a provider. Bedrock’s own region is aws_region in the provider configuration.Recommended: use IRSA for the Genesis service account, so no API key is stored at all — configure Bedrock with "auth_mode": "iam_role" and the pod’s role supplies the credential.
Genesis prepares GIT_PATH on startup and ensures repos/, worktrees/, and .genesis_runtime/ exist before shared Git-backed workflows use the mount. The shared group-write repair starts in the background so slow recursive permission fixes do not block container startup.

Inbound SCIM for AWS/EKS deployments

SCIM is optional. Enable it only when you want an external IdP to provision Genesis users and role membership automatically. Genesis acts as a SCIM 2.0 service provider, so the customer’s IdP pushes users and groups to Genesis at https://<genesis-host>/scim/v2. For the full SCIM documentation set, start with SCIM Inbound Provisioning. This section only covers the AWS/EKS deployment specifics.

Helm values for SCIM

Add the SCIM environment variables to your Helm values (or your secret-management workflow):
For the description of each variable, see the SCIM environment variables table.
Treat the SCIM bearer token like a password. Store it in your IdP’s secret field and your deployment secret-management system. Do not place it in screenshots, shared tickets, or shell history.

EKS rate limiting note

For multi-replica EKS deployments, enforce strict global SCIM rate limiting at ingress (for example, AWS WAF, NGINX rate-limit annotations, or an API gateway). The in-process per-token limiter is per-pod and is not sufficient as the only control across replicas. See SCIM Security for details.

AWS Identity Center caveat

AWS IAM Identity Center is primarily a SCIM target, not a SCIM client. It does not natively push outbound SCIM into arbitrary applications such as Genesis. For customers who want to drive Genesis from AWS Identity Store, the supported pattern is an AWS-hosted provisioner that reads Identity Store via AWS APIs and pushes SCIM into Genesis. See SCIM IdP Setup Guides — AWS-sourced provisioning for the architecture and step-by-step setup. For production rollouts that need direct outbound SCIM, use an IdP that natively pushes SCIM to custom applications, such as Okta, Azure Entra ID, or OneLogin.

Pilot, validation, and rollback

For the validation procedure, conformance harness, and rollback steps, see the SCIM Pilot Runbook.

Deployment Steps

1

Prepare Helm Chart

Genesis publishes the Helm chart to private OCI registries. After authenticating with the registry Genesis assigned to your deployment, pull and unpack the chart locally. If Genesis provided a .tgz archive instead, extract it and continue with the same chart directory layout:
Chart releases are published from main. On each GA release, Genesis updates appVersion in Chart.yaml to the promoted container version and patch-bumps the chart version automatically. To find the chart version for a specific GA release, query the GA_HELM_MAP repo variable: gh variable get GA_HELM_MAP -R genesis-computing-ai/genesis-agents | jq -r '.["vX.Y.Z"]'.
2

Create Values File

Create a genesis-values.yaml file with your configuration:
For IaC-managed deployments, keep cluster settings in genesis-values.yaml and copy the chart’s genesis-config-example.yaml to a separate genesis-config.yaml for providers, llm, embedding, auth, oauth_providers, and related app settings (including LiteLLM Proxy OAuth). Deploy with --set-file genesisConfig.content=genesis-config.yaml, or set genesisConfig.existingConfigMap when the file already exists as a ConfigMap. Genesis mounts the file at /app/.genesis/config/genesis-config.yaml, and explicit environment variables still take precedence over file-backed values.For Helm-driven RBAC on OIDC deployments, you can also pass --set-file groupRoleMapping.content=group-role-mapping.yaml and --set-file rbacSeedPolicy.content=rbac-policies.csv. See Deploying Genesis - Azure AKS with OIDC Authentication.
extraManifests defaults to []. Add complete Kubernetes resources here when you want Helm to deploy objects such as PodDisruptionBudget, NetworkPolicy, or PodMonitor alongside Genesis without forking the chart. Each list item is rendered as its own YAML document.
For NGINX Ingress Controller, use these annotations instead:
If you deploy with aws-eks/deploy-eks-auth0.sh, aws-eks/deploy-eks-oidc.sh, or aws-eks/deploy-eks-proxy-stack.sh, pin image.tag in the corresponding values-eks-*.yaml file. Those scripts pass ecr.enabled, ecr.registry, a fresh ecr.token, and image.pullPolicy=Always to Helm on every run, and they only pass --set image.tag when GENESIS_IMAGE_TAG is set in aws-eks/eks-config.env.
3

Deploy Genesis

4

Verify Deployment

5

Access Genesis

6

Configure the LLM Provider

Genesis starts without a provider and reports No suitable LLM found until one is configured. Configure it against the running pod, either from Config -> AI Providers in the UI or through the config API. Keep credentials in a request-body file — process arguments are visible to every process on the host and land in shell history.
Read each response before running the next call: these routes also answer 200 with success: false (for example when the provider already exists — repeat that call as a PUT). For the full endpoint reference, the card / llm-type naming table and the other providers, see Configuring a provider through the config API.
Some schema fields carry a default_env_var binding that seeds them from the environment while nothing is stored, so an image can come up pre-seeded. That is bootstrap only — it is not the way to configure a provider, and anything saved through the UI or the config API outranks it.

Kubernetes-Specific Configuration

StatefulSet

Genesis is deployed as a StatefulSet (not Deployment) to ensure:
  • Stable network identity (pod name: genesis-0)
  • Ordered, graceful deployment and scaling
  • Stable persistent storage (PVC name: genesis-data-genesis-0)

Persistent Volume Claim

The StatefulSet creates a PersistentVolumeClaim with:

Helm resource retention

Chart 0.3.70 and later annotate the Genesis Namespace, sandbox Namespace and RBAC, and both ServiceAccounts with helm.sh/resource-policy: keep. This keeps those objects in place when they move to a separate platform release in a later chart revision; it also means helm uninstall does not delete the Genesis Namespace. Delete the namespace explicitly when you intend to remove the deployment:
If sandbox access is enabled, the per-instance RoleBinding and ClusterRoleBinding are retained outside the Genesis Namespace as well. Turning sandbox.enabled off or changing sandbox.namespace does not revoke those bindings, and the old bindings can continue granting pod management and namespace-read access to the instance’s ServiceAccount. Revoke all bindings for an instance with:
After changing sandbox.namespace, run the cleanup command and then re-apply the Helm release so the current bindings are recreated. The command is also needed after a direct helm uninstall; cluster/scripts/teardown.sh runs it as part of teardown. Shared sandbox resources are retained for other instances and should only be removed with the corresponding shared-resource retirement workflow after the last instance is gone. The chart reserves helm.sh/resource-policy on retained objects. Do not set that key under serviceAccount.annotations or sandbox.podServiceAccount.annotations; Helm rejects the render and names the values path. A rollback to a chart version older than 0.3.70 removes the annotation from live objects, so re-check it after rollback:

Volume Mounts

The Genesis container mounts the persistent volume at multiple paths:

Health Checks

Genesis uses three Kubernetes probes so slow startups do not trigger liveness restarts before the app is ready:

Startup Probe

  • Endpoint: GET /api/health on port 8080
  • Period: 10 seconds
  • Timeout: 5 seconds
  • Failure threshold: 60 (up to ~10 minutes boot grace)
  • Gates liveness and readiness until startup completes (503 with progress, then 200)

Liveness Probe

  • Endpoint: GET /api/health/live on port 8080
  • Period: 30 seconds
  • Timeout: 5 seconds
  • Failure threshold: 3
  • Always 200 while the process is alive — does not require Genesis to be fully ready

Readiness Probe

  • Endpoint: GET /api/health on port 8080
  • Period: 10 seconds
  • Timeout: 5 seconds
  • Failure threshold: 3
  • Returns 503 while booting; 200 when ready to serve traffic
When Genesis sits behind OAuth2 Proxy (authProvider: "proxy", for example via aws-eks/deploy-eks-proxy-stack.sh), configure the proxy so GET health probes and the dashboard healthcheck endpoint bypass authentication. The bundled aws-eks/oauth2-proxy-values-eks-example.yaml includes:
cookie_secure = true is correct when HTTPS terminates at the ingress controller or external load balancer, even if the OAuth2 Proxy chart ingress has tls: []. Routes are GET-scoped and anchored so other /api/health/* paths (errors, threads, slots, and so on) remain authenticated. cookie_csrf_per_request isolates concurrent login flows and prevents invalid-CSRF 403 responses.

Auth-exempt health Ingress for auth_request deployments

If an NGINX ingress uses nginx.ingress.kubernetes.io/auth-url and nginx.ingress.kubernetes.io/auth-signin, an external load balancer or uptime monitor can receive the proxy’s login redirect instead of Genesis health status. Enable the chart’s second, unauthenticated health Ingress for this deployment pattern:
ingress.authExemptHealth.enabled defaults to false. When enabled, the chart renders <release>-health on the same hosts and TLS configuration as the main Ingress, routing these exact paths to the FastAPI Service: /api/health, /api/health/startup, /api/health/live, and /api/healthcheck, each with and without a trailing slash. ingress.authExemptHealth.pathType defaults to Exact; keep it that way so the authenticated /api/health/detailed and /api/health/slots routes are not exposed. The backend port defaults to service.ports.fastapi (8080) and can be overridden with ingress.authExemptHealth.port. The health Ingress does not inherit ingress.annotations. Put only its own annotations under ingress.authExemptHealth.annotations. For the AWS Load Balancer Controller, add the ALB group annotation there so both Ingress resources use the same load balancer:

Graceful shutdown (podLifecycle)

The chart sets a preStop sleep and a pod-level termination grace period so ingress load balancers can drain in-flight requests before Genesis receives SIGTERM.
Increase both values if your ingress controller or ALB target deregistration is slower than the defaults.

Extension containers (extraContainers, extraInitContainers)

Pass additional Kubernetes container specs through the chart without forking templates. extraInitContainers is rendered when customCACerts is enabled, when extraInitContainers is non-empty, or both.

Resource Requirements

Default resource requests:
No resource limits are set by default to avoid OOM kills. Adjust based on your workload.

Vertical Pod Autoscaler (Optional)

Genesis runs as a single-replica StatefulSet backed by a ReadWriteOnce persistent volume, so horizontal pod autoscaling is not supported. If you want automatic right-sizing for resources.requests, enable the chart’s optional Vertical Pod Autoscaler (VPA) support instead. Prerequisites:
  1. metrics-server must be installed in the cluster.
  2. Install a VPA controller. From the Helm chart directory, you can use the bundled helper:
vpa.updateMode: "Auto" can cause brief downtime because the single Genesis pod must be evicted and recreated to apply new resource requests. Start with vpa.updateMode: "Off" unless brief resizes are acceptable in your environment.
Example: recommendation-only mode
Apply the updated values and then verify the VPA object:
The examples on this page use Helm release name genesis in namespace genesis. If you deploy with a different release name or namespace, check the generated VPA name with kubectl get vpa -n <namespace> before describing it.

AWS-Specific Considerations

IAM Roles for Service Accounts (IRSA)

For production, configure IRSA so the Genesis pod can access AWS services without storing long-lived credentials. This is the recommended pattern for AWS Bedrock IAM authentication.
  1. Create IAM Role with Bedrock permissions
  2. Create Service Account with role annotation:
  1. Configure the Helm release to use the annotated service account:
Genesis detects the AWS web-identity environment variables injected by EKS and can use IAM Role authentication for Bedrock without an API key.

Agent sandbox execution (optional)

Genesis agents can run long-running Python jobs, shell commands, and scripts in isolated Kubernetes pods instead of the application container’s local Docker runtime. On EKS, enable this through Helm sandbox values. The Genesis ServiceAccount needs IRSA for the Genesis pod itself (Bedrock, Secrets Manager, and other control-plane AWS calls). Sandbox workload pods run in a separate namespace and need their own ServiceAccount with IRSA when agents call AWS APIs from inside the sandbox (for example boto3, S3, or Bedrock). When sandbox.enabled is true, the chart grants the release ServiceAccount pod lifecycle permissions in the sandbox namespace (create, exec, delete, get, list, watch on pods and pod logs) and read access to the sandbox namespace object itself. Example: EKS lab instance
serviceAccount.create must be true so the chart can bind sandbox RBAC to the Genesis ServiceAccount. Set sandbox.podServiceAccount.create to true and annotate sandbox.podServiceAccount.annotations with an IRSA role trusted for <sandbox.namespace>:<sandbox.podServiceAccount.name> so sandbox pods receive AWS credentials. When starting sandboxes, pass the same name via backend_options.service_account (or spec.k8s.service_account) — for example genesis when using the default sandbox.podServiceAccount.name. The configured defaultBackend maps to sandbox.default_backend in Genesis APP_CONFIG. Agents that call sandbox_action without an explicit backend (or with backend="local") use that operator default instead of local Docker. See sandbox_action.

Bootstrap secrets via config.extraEnv (AWS Secrets Manager)

On EKS with IRSA, you can reference AWS Secrets Manager secrets in Helm config.extraEnv. Genesis resolves aws_* shorthand or full ARNs at pod startup using the pod’s IRSA credentials — no init container, Kubernetes Secret, or CSI driver is required for the reference itself. Grant secretsmanager:GetSecretValue on the secret to the IRSA role (same as Custom CA Certificates and External Secrets). After you rotate a value in AWS Secrets Manager, restart the StatefulSet so the pod re-resolves it:
Example: Databricks API key on EKS (chart file values_databricks_llm.yaml):
Short form is also supported:
For the same Databricks LLM deployment, you can instead pass the token at install time (--set secrets.databricksApiKey="dapi...") or mount a pre-created Kubernetes secret via secrets.existingSecretDatabricksApiKey. The bundled values_databricks_llm.yaml sample leaves config.modelName, config.harvesterModel, config.fastModelName and config.embeddingModel empty, so the catalog default for each role applies: databricks-claude-opus-5-5 for the main and harvester models, databricks-claude-haiku-4-5 for the fast model, and databricks-gte-large-en for embeddings. Set any of these fields to seed a different model when no corresponding model is already saved in AppConfig. All three forms are default_env_var bootstrap seeding: the value is captured only while nothing is stored for the field, and it is outranked by anything saved through the UI or the config API. Seeding a credential does not configure or activate the provider — do that from Configuring a provider through the config API. Example: Master encryption key from AWS Secrets Manager (leave genesisMasterKey empty and resolve via IRSA at pod startup):
Grant secretsmanager:GetSecretValue on the referenced secret to the IRSA role. If GENESIS_MASTER_KEY is also set in the pod environment, it takes precedence over GENESIS_MASTER_KEY_REF. See Master encryption key for the full resolution order.
Some non-production deployments grant Bedrock permissions to the EKS node role. That is acceptable for isolated testing, but IRSA is preferred for production because it scopes permissions to the Genesis service account rather than all pods on a node.

Custom CA Certificates from AWS Secrets Manager

If Genesis must connect to internal HTTPS endpoints signed by a private CA, the Helm chart can fetch CA certificates from AWS Secrets Manager during pod startup and merge them into the container trust store. It uses the same secret reference syntax documented in External Secrets.
Grant secretsmanager:GetSecretValue on each referenced secret to the IRSA role. When enabled, the chart mounts the merged CA bundle and sets REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE, SSL_CERT_FILE, GIT_SSL_CAINFO, and NODE_EXTRA_CA_CERTS so Python HTTP clients, curl, Git clone/fetch, and Node.js trust the same custom CAs as API calls.
The fetch-ca-certs init container uses the image’s /etc/ssl/certs/ca-certificates.crt system bundle to contact Secrets Manager. In that container only, the chart overrides SSL_CERT_FILE, REQUESTS_CA_BUNDLE, CURL_CA_BUNDLE, and AWS_CA_BUNDLE from config.extraEnv; this prevents paths to a custom bundle that is not available until Genesis starts from breaking certificate retrieval. The Genesis container continues to use the merged custom CA bundle.
If you already distribute CA files through Kubernetes Secrets or a CSI driver, continue using extraVolumes and extraVolumeMounts to mount them directly instead of customCACerts.
When you rotate a certificate in Secrets Manager, restart the StatefulSet so the pod rebuilds the trust bundle:

EBS Volume Encryption

EBS volumes created by the EBS CSI driver on EKS are encrypted by default if your cluster has encryption enabled. Verify:

VPC Configuration

Ensure your EKS cluster nodes have:

Troubleshooting

Common issues:
  • Image pull errors: Check network connectivity to the container registry provided by Genesis
  • PVC issues: Verify EBS CSI driver is installed and storage class exists
  • Resource constraints: Check node resources
For AWS Load Balancer Controller:
For NGINX Ingress:

Upgrading Genesis

When a new Genesis version is released:
The StatefulSet will perform a rolling update, and the persistent volume will be reattached to the new pod.
For values-driven upgrades, update image.tag in the relevant values-eks-*.yaml file and rerun the matching aws-eks/deploy-eks-*.sh helper. If you use aws-eks/update-eks-image.sh or aws-eks/update-eks-helm.sh, both helpers require ECR_REGISTRY in aws-eks/eks-config.env, default to v1.18.2 when no tag argument is provided, refresh the ECR token on every run, and perform a kubectl rollout restart after helm upgrade so EKS nodes pull the updated image and pull secret.
Purely numeric image.tag values (for example ECR build IDs) are coerced to strings automatically in chart 0.3.55 and later. On older charts, quote the tag in YAML or use --set-string image.tag=... instead of --set.

Uninstalling

Deleting the PVC will permanently remove all Genesis data including the database, git repositories, and uploaded files.

Additional Resources

AWS EKS Documentation

Official Amazon EKS documentation

EBS CSI Driver

EBS CSI Driver setup guide

AWS Load Balancer Controller

AWS Load Balancer Controller documentation