External Secrets - Genesis Computing

Overview

Genesis can resolve secrets stored in your cloud provider’s secrets manager at runtime. Instead of entering a raw credential (password, token, API key) in a connection form, you enter a reference to the secret in your vault. Genesis stores the reference securely, and resolves it to the real credential at runtime via chain resolution.

The external secret reference is stored encrypted in Genesis like any other credential. The actual secret value from AWS/GCP is resolved in-memory only — it is never stored in the Genesis database, never logged, and never sent to the LLM.

When to use external secrets

Your organization requires credentials to remain in a managed vault (e.g., AWS Secrets Manager, GCP Secret Manager)
You have existing secret rotation workflows and want Genesis to pick up rotated values automatically
Compliance policies prohibit storing credentials outside of approved key management systems

Supported providers

AWS Secrets Manager

Uses the boto3 default credential chain — IAM roles on EKS, EC2, ECS, and Lambda.

GCP Secret Manager

Uses Application Default Credentials — Workload Identity on GKE, service accounts on Cloud Run.

Reference syntax

Instead of entering a credential value directly, enter a secret reference string. Genesis stores the reference (for schema is_secret fields, encrypted in APP_CONFIG) and resolves it to the actual value when the setting or connection is used. Bare aws_*, gcp_*, and arn:aws:secretsmanager:* references stored directly in APP_CONFIG — for example in genesis-config.yaml or Advanced Settings — are resolved at runtime without wrapping them in a secret_* Genesis secret. The same storage rules apply whether you save through Advanced Settings, genesis-config.yaml, config profiles, or the setup APIs — references are never baked in at write time. Plain text entered for a secret schema field is migrated once into the Genesis secret store; the config row keeps the field’s default_secret reference (or NULL, which resolves to that default at read time).

Genesis internal secrets

Reference	Resolves to
`secret_my-api-key`	Value from the Secrets store (encrypted in the Genesis database)

Names must start with secret_. A Genesis secret’s stored value may itself be an aws_* or gcp_* reference; Genesis resolves that chain at runtime.

Environment variables

Reference	Resolves to
`env_OPENAI_API_KEY`	`OPENAI_API_KEY` from the process environment
`${DATABASE_URL}`	`DATABASE_URL` from the process environment

Genesis stores env_* and ${VAR} references as-is. Each resolution reads the current environment, so updating the variable in the deployment (Kubernetes Secret, Helm extraEnv, etc.) takes effect on the next use without a Genesis restart. This is separate from schema fields that declare default_env_var: non-secret fields read that env var live on every config snapshot and Genesis also exports resolved values into the environment at startup; secret fields instead capture the env value into their default managed secret (secret_<path>) once at startup when unset. See Advanced Settings — Secret-valued settings.

Master encryption key

Genesis encrypts credentials in the database with AES-256-GCM. The master encryption key is resolved once at startup using this priority order:

GENESIS_MASTER_KEY environment variable (always takes precedence)
GENESIS_MASTER_KEY_REF external secret reference (aws_* or gcp_*, resolved via IRSA on EKS or Application Default Credentials on GKE)
Snowflake native app secret (when SPCS_MODE=TRUE)
Databricks secret (when DBX_MODE=TRUE, except in a Databricks App)
Default development key (local development only — never used in SPCS/DBX mode or a Databricks App)

For Genesis Desktop, the shell creates a random master key for each install, stores it in the operating system’s credential store (the login keychain on macOS), and supplies it to the backend as GENESIS_MASTER_KEY. On startup, Desktop moves stored secrets that the development key can open onto the install key. Secrets it cannot open are left unchanged. Once migration is complete, the development key is no longer accepted for that install. After migration, if Desktop cannot read its key from the credential store, it shows an encryption-key page. Quit and reopen Genesis, and choose Always Allow if the operating system asks for keychain access; existing saved credentials are kept unchanged. If the keychain key has changed, Desktop reports the number of unreadable saved credentials and asks the person to reconnect affected connections in Config -> Connections. The credentials are not deleted or overwritten by this check. Desktop records when that notice is dismissed; see Genesis Desktop saved credential notice for the status field and acknowledgment endpoint. A Databricks App must bind its own secret resource to GENESIS_MASTER_KEY (or provide GENESIS_MASTER_KEY_REF); it does not read or create a Databricks workspace secret and never falls back to the development key. The private App pilot’s resource setup is in Deploying Genesis on Databricks — Private full Genesis App pilot. For App key rotation and automatic re-sealing of stored secrets, see Rotate the master key. For Kubernetes deployments via Helm, supply the master key on first install using one of:

secrets.genesisMasterKey — inline value (chart stores it in a Kubernetes Secret as GENESIS_MASTER_KEY)
secrets.existingSecretGenesisMasterKey — reference an existing Kubernetes Secret
config.extraEnv.GENESIS_MASTER_KEY_REF — aws_* or gcp_* reference resolved at pod startup (leave genesisMasterKey empty)

Example on EKS with IRSA:

secrets:
  genesisMasterKey: ""

config:
  extraEnv:
    GENESIS_MASTER_KEY_REF: "aws_prod/genesis-keys#master_key"

serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT:role/genesis-secrets-reader"

The workload identity (IRSA on EKS, Workload Identity on GKE) must have permission to read the referenced secret. If GENESIS_MASTER_KEY_REF is set but resolution fails, Genesis fails to start. If GENESIS_MASTER_KEY is also set (for example from a Kubernetes Secret), it overrides GENESIS_MASTER_KEY_REF. When a request needs a key that is not configured, a browser GET request that accepts text/html receives a setup page with provider-specific guidance and HTTP 503, rather than a raw JSON error. Other methods and clients that do not accept HTML continue to receive the JSON error response. If Snowflake cannot read an existing key temporarily, the page says to retry and not to generate a replacement; stored credentials depend on the existing key.

AWS Secrets Manager

Reference	Resolves to
`aws_my-database-creds`	Full secret value (latest version)
`aws_my-database-creds#password`	The `password` field from a JSON secret
`aws_my-database-creds#password@AWSPREVIOUS`	Previous rotation version, `password` field
`aws_my-database-creds@AWSCURRENT`	Explicit staging label
`arn:aws:secretsmanager:us-east-1:123456:secret:prod/db`	Full ARN reference

GCP Secret Manager

Reference	Resolves to
`gcp_my-api-key`	Full secret value (latest version)
`gcp_my-api-key#token`	The `token` field from a JSON secret
`gcp_my-api-key@3`	Specific version number
`gcp_projects/my-project/secrets/my-key`	Explicit project path

Syntax rules

Prefix identifies the backend (secret_, env_, aws_, gcp_, or vault_* when supported)
# extracts a JSON property from composite AWS/GCP secrets
@ selects a specific version or staging label (AWS/GCP)
For env_*, the suffix after env_ is the environment variable name (for example env_OPENAI_API_KEY → OPENAI_API_KEY)
${VAR} must be the entire string (no surrounding text)
Everything between the prefix and #/@ is the external secret name

AWS setup

Genesis resolves AWS secrets using the boto3 default credential chain. On EKS this means IRSA (IAM Roles for Service Accounts); on EC2/ECS it uses instance profiles or task roles. The steps below cover the full setup.

The examples below use the AWS CLI. If you use Terraform, CloudFormation, or the AWS Console, apply the same policy and trust policy documents using your preferred tool.

EKS
EC2 / ECS

Step 1: Verify the OIDC provider is enabled

IRSA requires an OpenID Connect (OIDC) provider associated with your EKS cluster. Most clusters created via eksctl or Terraform have this enabled by default.

aws eks describe-cluster --name YOUR_CLUSTER \
  --query "cluster.identity.oidc.issuer" --output text

If this returns an OIDC issuer URL, confirm that the corresponding IAM OIDC provider exists in your account:

aws iam list-open-id-connect-providers | grep $(aws eks describe-cluster \
  --name YOUR_CLUSTER --query "cluster.identity.oidc.issuer" --output text \
  | sed 's|https://||')

If no match is returned, create the IAM OIDC provider:

eksctl utils associate-iam-oidc-provider --cluster YOUR_CLUSTER --approve

Note your OIDC ID — it is the last segment of the issuer URL (e.g., https://oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE → EXAMPLED539D4633E53DE1B71EXAMPLE).

Step 2: Create an IAM policy

Create a policy that grants secretsmanager:GetSecretValue on the secrets you want Genesis to access. The Resource ARN must match your secret names in AWS Secrets Manager — Genesis does not require any specific naming convention.Save the policy document:

cat > genesis-secrets-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "secretsmanager:GetSecretValue",
      "Resource": [
        "arn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:prod/db-creds-*",
        "arn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:prod/api-token-*"
      ]
    }
  ]
}
EOF

Create the policy:

aws iam create-policy \
  --policy-name genesis-secrets-reader-policy \
  --policy-document file://genesis-secrets-policy.json

Replace REGION and ACCOUNT_ID with your values.
The Resource ARN pattern must match your secret names. AWS appends a random 6-character suffix to every secret ARN, so always end patterns with -* (e.g., prod/db-creds-* matches a secret named prod/db-creds).
Use a wildcard like prod/* to grant access to all secrets under a path prefix, or list individual secret ARNs for tighter scoping.

Step 3: Create an IAM role with an OIDC trust policy

Save the trust policy document:

cat > genesis-secrets-trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/oidc.eks.REGION.amazonaws.com/id/OIDC_ID"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.eks.REGION.amazonaws.com/id/OIDC_ID:aud": "sts.amazonaws.com",
          "oidc.eks.REGION.amazonaws.com/id/OIDC_ID:sub": "system:serviceaccount:NAMESPACE:genesis"
        }
      }
    }
  ]
}
EOF

Replace the placeholders:

ACCOUNT_ID — your AWS account ID
REGION — the AWS region of your EKS cluster (e.g., us-east-1)
OIDC_ID — the OIDC provider ID from Step 1
NAMESPACE — the Kubernetes namespace where Genesis is deployed (e.g., genesis)

The service account name is genesis by default (set via serviceAccount.name in the Helm chart).

Create the role and attach the policy:

aws iam create-role \
  --role-name genesis-secrets-reader \
  --assume-role-policy-document file://genesis-secrets-trust-policy.json

aws iam attach-role-policy \
  --role-name genesis-secrets-reader \
  --policy-arn arn:aws:iam::ACCOUNT_ID:policy/genesis-secrets-reader-policy

Step 4: Configure the Genesis Helm chart

In your Helm values.yaml, enable the service account and set the IRSA role annotation:

serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT_ID:role/genesis-secrets-reader"

Then deploy (or upgrade) the release:

helm upgrade --install genesis ./genesis -f values.yaml

The chart creates a Kubernetes service account named genesis with the IRSA annotation. EKS automatically injects AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN into the Genesis pod, giving it access to assume the IAM role.

For Helm bootstrap environment variables (for example DATABRICKS_API_KEY in values_databricks_llm.yaml), you can set config.extraEnv to an aws_* reference or full Secrets Manager ARN. Genesis resolves it at pod startup; restart the StatefulSet after rotating the secret in AWS. See Genesis on AWS — Bootstrap secrets via config.extraEnv.

Step 5: Verify

Confirm the pod received IRSA credentials:

kubectl exec -n NAMESPACE genesis-0 -- env | grep -E 'AWS_ROLE_ARN|AWS_WEB_IDENTITY_TOKEN_FILE'

You should see both AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE. If either is missing, verify the service account annotation and restart the pod.Test that Genesis can read a secret from AWS. First, create a test secret:

aws secretsmanager create-secret \
  --name prod/db-creds \
  --secret-string '{"username":"admin","password":"s3cret"}'

Then in Genesis, create a connection and enter aws_prod/db-creds#password in the password field. Test the connection — Genesis should resolve the reference and connect successfully.To clean up the test secret:

aws secretsmanager delete-secret \
  --secret-id prod/db-creds --force-delete-without-recovery

Step 1: Create an IAM policy

Save the policy document:

cat > genesis-secrets-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "secretsmanager:GetSecretValue",
      "Resource": [
        "arn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:prod/db-creds-*",
        "arn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:prod/api-token-*"
      ]
    }
  ]
}
EOF

Create the policy:

aws iam create-policy \
  --policy-name genesis-secrets-reader-policy \
  --policy-document file://genesis-secrets-policy.json

Replace REGION and ACCOUNT_ID with your values.
AWS appends a random 6-character suffix to every secret ARN, so always end patterns with -*.
Use a wildcard like prod/* to grant access to all secrets under a path prefix.

Step 2: Attach the policy

EC2: Attach the policy to the IAM instance profile associated with the Genesis EC2 instance.
ECS: Attach the policy to the ECS task role (not the task execution role) for the Genesis task definition.

Step 3: Verify

Test that Genesis can read a secret from AWS. First, create a test secret:

aws secretsmanager create-secret \
  --name prod/db-creds \
  --secret-string '{"username":"admin","password":"s3cret"}'

aws secretsmanager delete-secret \
  --secret-id prod/db-creds --force-delete-without-recovery

GCP setup

Prerequisites

Genesis must be running on GCP infrastructure with access to Secret Manager. This works automatically on GKE and Cloud Run when the service account has the correct IAM binding.

Step 1: Grant Secret Manager access

Grant the Secret Manager Accessor role on the secrets Genesis needs:

gcloud secrets add-iam-policy-binding my-secret \
  --member="serviceAccount:GENESIS_SA@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

Replace GENESIS_SA and PROJECT_ID with your service account name and GCP project ID.

Step 2: Configure workload identity

GKE (Workload Identity)
Cloud Run

Enable Workload Identity on your GKE cluster
Bind the Kubernetes service account to the GCP service account:

gcloud iam service-accounts add-iam-policy-binding \
  GENESIS_SA@PROJECT_ID.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]"

Annotate the Kubernetes service account:

kubectl annotate serviceaccount KSA_NAME \
  iam.gke.io/gcp-service-account=GENESIS_SA@PROJECT_ID.iam.gserviceaccount.com

Step 3: Verify

Store a test secret in GCP Secret Manager
In Genesis, create a connection and enter gcp_your-test-secret in a credential field
Test the connection — Genesis should resolve the reference and connect successfully

Using external secrets in connections

External secret references can be entered anywhere you would normally enter a credential value. Simply type the reference string instead of the raw credential:

Database connections: Enter aws_prod/postgres-creds#password in the password field
GitHub PAT: Enter gcp_github-pat in the token field
API integrations: Enter aws_my-api-key in the API key field
Any credential field in the Connections UI accepts these reference strings

Genesis handles everything automatically — the reference is stored encrypted, and the real credential is fetched from your vault when the connection is used. No additional configuration is needed in the Genesis UI.

Using external secrets in LLM provider configuration

The same reference syntax works in Config → AI Providers and in genesis-config.yaml under providers.* secret fields (for example providers.litellm_proxy.connection.api_key.proxy_api_key, providers.litellm_bedrock.connection.api_key.aws_bearer_token, and OAuth client_secret values). Enter aws_*, gcp_*, or vault_* references instead of plaintext API keys when bootstrapping Helm deployments or saving provider credentials in the UI. Genesis preserves external references through the App Config → legacy LLM runtime config bridge and resolves them with resolve_secret() when chat, embedding, or provider test calls run. Managed secret_* references are included only when the backing Genesis secret exists; external refs are always included because they resolve outside the Genesis SECRETS table. See Setup AI Model — Infrastructure-managed deployments.

vault_* references are recognized in configuration and the LLM pipeline, but HashiCorp Vault resolution is not yet implemented — use aws_* or gcp_* for production external-secret backends today.

Caching and rotation

Genesis caches resolved AWS/GCP external secrets for 5 minutes by default
Configure the cache TTL with the GENESIS_EXTERNAL_SECRET_CACHE_TTL_SECONDS environment variable
When you rotate a secret in AWS or GCP, Genesis picks up the new value within the cache TTL window
env_* and ${VAR} references are not cached — each resolution reads the current process environment, so pod/env updates take effect without a Genesis restart
secret_* references pick up rotated values from the Genesis secret store on the next resolution (no restart)
Set the external-secret cache TTL to 0 to disable AWS/GCP caching (every resolution hits the cloud API)

Troubleshooting

Error message	Cause	Fix
”AWS authentication failed”	Genesis pod/instance lacks IAM permissions	EKS: Check that `serviceAccount.create: true` and the `eks.amazonaws.com/role-arn` annotation are set in Helm values, and that the IAM role trust policy matches the cluster OIDC provider, namespace, and service account name. EC2/ECS: Verify the instance profile or task role has the Secrets Manager policy.
”AWS secret ‘X’ not found”	Secret name does not match	Check the secret name in the AWS Secrets Manager console
”property ‘X’ not found in secret JSON”	The JSON property path is incorrect	Verify the JSON structure of the secret value
”GCP authentication failed”	Workload Identity is not configured	Check the service account binding and IAM role
”GCP secret ‘X’ not found”	Wrong project or secret name	Verify with `gcloud secrets list`
”cannot infer GCP project ID”	Short-form key used without ADC project	Use the full path: `gcp_projects/PROJECT/secrets/NAME`
`GENESIS_MASTER_KEY_REF is set to 'X' but resolution failed`	Pod lacks IAM/ADC access or secret does not exist	Verify the `aws_` / `gcp_` reference, grant `secretsmanager:GetSecretValue` or Secret Manager accessor on the referenced secret, and confirm IRSA/Workload Identity is configured.

Access control models

In both models below, your credentials always stay in your cloud vault — Genesis only stores an encrypted reference.

Service identity (current)

Genesis authenticates to AWS/GCP using its own workload identity (IAM role on EKS/EC2/ECS, or GCP service account). A single IAM policy controls which secrets Genesis can read.

Genesis RBAC controls which users and agents can trigger secret resolution
Cloud IAM controls which secrets the Genesis workload can fetch
Both layers must pass for a secret to be resolved

This is the default model and requires no per-user IAM setup.

Per-user IAM role assumption (coming soon)

For organizations that require each user’s own cloud identity to authenticate to the secrets manager, Genesis can assume an IAM role on behalf of the user. In this model, your cloud IAM policies control which secrets each user can access — Genesis acts as a pass-through.

Per-user IAM role assumption requires additional configuration on your cloud account (IAM roles, trust policies, user-to-role mapping). Contact your Genesis representative to discuss setup.

Security

External secret references are stored encrypted in Genesis (like any credential value)
Actual secret values from AWS/GCP are resolved in-memory only — never written to the Genesis database
Values are masked in all Genesis logs (****)
Values are never included in LLM context or prompts
Sandboxed tools do not inherit ambient cloud credentials such as AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, or AWS_SESSION_TOKEN; pass cloud auth through a Genesis secret, external secret reference, or saved connection instead of relying on host environment variables
Genesis RBAC controls who can access the Genesis secret containing the reference
Cloud IAM controls which secrets the Genesis workload can fetch from AWS/GCP
Genesis logs which user or agent resolved which external secret reference (without the value) in the SECRETS_AUDIT table. For SQLite-specific delivery and retry behavior, see SQLite audit writes.

​Overview

​When to use external secrets

​Supported providers

AWS Secrets Manager

GCP Secret Manager

​Reference syntax

​Genesis internal secrets

​Environment variables

​Master encryption key

​AWS Secrets Manager

​GCP Secret Manager

​Syntax rules

​AWS setup

​Step 1: Verify the OIDC provider is enabled

​Step 2: Create an IAM policy

​Step 3: Create an IAM role with an OIDC trust policy

​Step 4: Configure the Genesis Helm chart

​Step 5: Verify

​Step 1: Create an IAM policy

​Step 2: Attach the policy

​Step 3: Verify

​GCP setup

​Prerequisites

​Step 1: Grant Secret Manager access

​Step 2: Configure workload identity

​Step 3: Verify

​Using external secrets in connections

​Using external secrets in LLM provider configuration

​Caching and rotation

​Troubleshooting

​Access control models

​Service identity (current)

​Per-user IAM role assumption (coming soon)

​Security

​Related pages

Overview

When to use external secrets

Supported providers

Reference syntax

Genesis internal secrets

Environment variables

Master encryption key

AWS Secrets Manager

GCP Secret Manager

Syntax rules

AWS setup

Step 1: Verify the OIDC provider is enabled

Step 2: Create an IAM policy

Step 3: Create an IAM role with an OIDC trust policy

Step 4: Configure the Genesis Helm chart

Step 5: Verify

Step 1: Create an IAM policy

Step 2: Attach the policy

Step 3: Verify

GCP setup

Prerequisites

Step 1: Grant Secret Manager access

Step 2: Configure workload identity

Step 3: Verify

Using external secrets in connections

Using external secrets in LLM provider configuration

Caching and rotation

Troubleshooting

Access control models

Service identity (current)

Per-user IAM role assumption (coming soon)

Security

Related pages