State Snapshots and Migration

​

Overview

​

What a snapshot contains

s3://<bucket>/<prefix>/<instance>/<snapshot-id>/     (or gs://...)
  db/                 → /app/.genesis/db   (genesis.db and its neighbours)
  bot_storage/        → /app/bot_storage
  bot_git/            → /app/bot_git
  reports/            per-table copy report (Snowflake sources); not signed, not restored
  manifest.json       written last; its presence marks the snapshot complete

​

Before you start

• A bucket. Private. On S3, with KMS encryption; to restore in a different AWS account, use a customer-managed KMS key (kms_key_id) and grant the target account use of it, because the AWS-managed aws/s3 key cannot be shared across accounts. On GCS, with uniform bucket-level access (the default for new buckets) and public access prevention; kms_key_id does not apply.
• Credentials for the source, GCS. The exporter lists, reads, writes and deletes objects (it finds the previous snapshot, copies unchanged files from it and removes a failed attempt), so grant roles/storage.objectAdmin on the bucket; roles/storage.objectCreator alone makes snapshots fail. On GKE, use the pod’s Workload Identity: annotate the chart’s service account with iam.gke.io/gcp-service-account for a Google service account holding that role (or bind the Kubernetes service account directly). On Snowflake SPCS, create a GCP connection in Genesis with that service account’s key, and name it in the request as gcp_connection. A restore only reads, so roles/storage.objectViewer is enough for the restoring cluster.
• Credentials for the source, S3. These are AWS credentials. On EKS, the pod’s identity (IRSA) with write access to the prefix. On other clusters (GKE, AKS), GKE Workload Identity and Azure identities grant nothing on AWS: either trust the cluster’s OIDC issuer in AWS IAM and give the pod AWS_ROLE_ARN and a projected token at AWS_WEB_IDENTITY_TOKEN_FILE, or name a Genesis AWS connection in the request. On Snowflake SPCS there is no pod identity: create an AWS connection in Genesis with an access key that can write to the prefix, and name it in the request.
• Network egress from SPCS. The Native App’s external access integration must allow the bucket’s endpoints: for S3, <bucket>.s3.<region>.amazonaws.com; for GCS, storage.googleapis.com and oauth2.googleapis.com.
• The master key. The target must use the same master key as the source, or it cannot decrypt any stored credential. Note the source’s key before migrating; on SPCS it is the Snowflake secret behind CORE.GET_ENCRYPTION_KEY().

​

Take a snapshot

curl -X POST https://<genesis>/api/admin/state-snapshots \
  -H "Authorization: Bearer <token>" -H "Content-Type: application/json" \
  -d '{
        "destination": "s3://genesis-snapshots/prod/<instance>",
        "region": "us-west-2",
        "kms_key_id": "arn:aws:kms:us-west-2:111122223333:key/...",
        "aws_connection": "snapshots",
        "confirm": "<instance name>"
      }'

curl https://<genesis>/api/admin/state-snapshots/current

curl https://<genesis>/api/admin/state-snapshots -H "Authorization: Bearer <token>"

​

Restore onto Kubernetes

restore:
  from: "s3://genesis-snapshots/prod/<instance>"   # newest complete snapshot
  # or a specific one: s3://genesis-snapshots/prod/<instance>/<snapshot-id>
  # or on GCS: gs://genesis-snapshots/prod/<instance> (region is S3-only)
  region: "us-west-2"

secrets:
  existingSecretGenesisMasterKey:
    secretName: genesis-master-key     # the SOURCE instance's key
    secretKey: GENESIS_MASTER_KEY

serviceAccount:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<account>:role/genesis-snapshot-reader

• the manifest signature does not verify under the current or bound previous key, or a key named by requires_previous_key_fingerprint is not bound here;
• the snapshot comes from a newer Genesis release than the image (a development build, which has no release number, accepts a snapshot from any release);
• the manifest lists a path outside the volume’s trees, or the same path twice;
• any file is missing, unexpected or does not match its checksum, or the database fails its integrity check.

python3 -m scripts.uv.import_state_runner \
  --from s3://genesis-snapshots/prod/<instance> --region us-west-2 \
  --target-root /genesis-data --runtime-root /app --force --yes

​

Migrating from Snowflake SPCS

1. Upgrade the Native App to a release with state snapshot support, and allow the bucket’s endpoints in its external access integration.
2. In Genesis on SPCS, create a connection for the snapshot bucket: a GCP connection whose service account holds roles/storage.objectAdmin on a GCS bucket (the path to GKE), or an AWS connection that can list, read, write and delete under an S3 prefix.
3. Record the SPCS master key and store it as a Kubernetes Secret in the target cluster.
4. Take a snapshot with gcp_connection or aws_connection set. The exporter converts the Snowflake metadata tables into the Kubernetes SQLite database, reading every table as of the same instant through Time Travel. A table with no history at that instant (created after it, or kept without Time Travel retention) is read as it is at copy time instead. reports/tables.json, written beside the trees, lists every table, its row count, whether it was read at the shared instant (point_in_time), duplicate keys it collapsed (Snowflake does not enforce keys), and any column that exists on only one side. A row SQLite would reject for any other reason fails the snapshot rather than being dropped. The report is for the operator: it is not in the signed manifest, and a restore neither downloads nor checks it.
5. Install the Helm chart with restore.from pointing at the snapshot and the SPCS master key configured.
6. Configure authentication for the Kubernetes install (Auth0 or another OIDC provider). Snowflake sign-in does not carry over; users sign in again and roles are granted to their new identities.
7. Replace connections that relied on the SPCS service’s own Snowflake session with regular Snowflake connections (key pair or OAuth).