> ## Documentation Index > Fetch the complete documentation index at: https://docs.genesiscomputing.com/llms.txt > Use this file to discover all available pages before exploring further. # State Snapshots and Migration > Snapshot a Genesis instance to S3 or Google Cloud Storage and restore it onto a Kubernetes install: backups, cluster moves, and migration from Snowflake SPCS. ## Overview A state snapshot is a complete copy of a Genesis instance: its metadata database, its bot storage (uploads, threads, wiki, context graph, scheduler, apps, datasets) and its git repositories. It is written to an S3 or Google Cloud Storage (GCS) bucket in the layout of the Genesis Helm chart's volume, so it can be restored onto any Kubernetes install of the chart. The same mechanism serves three purposes: | Purpose | Source | Target | | - | - | - | | Backup and point-in-time restore | Kubernetes install | The same install | | Moving clusters (for example EKS → GKE) | Kubernetes install | A new Kubernetes install | | Migrating off Snowflake | Snowflake SPCS Native App | A Kubernetes install | Taking a snapshot pauses the instance: the server stops, the snapshot is copied while nothing writes to the state, and the server starts again on its own. Repeat snapshots to the same location upload only files that changed and copy the rest inside the bucket. ## What a snapshot contains Each snapshot is a folder of ordinary objects under the location you choose: ```text theme={null} s3:////// (or gs://...) db/ → /app/.genesis/db (genesis.db and its neighbours) bot_storage/ → /app/bot_storage bot_git/ → /app/bot_git reports/ per-table copy report (Snowflake sources); not signed, not restored manifest.json written last; its presence marks the snapshot complete ``` The manifest lists every file of the three trees with its size and SHA-256, every directory, the file modes, the source's version, a fingerprint of the source's master encryption key, and a signature made with that key. A snapshot without a manifest never finished and is ignored. A snapshot holds every credential Genesis has stored, still encrypted with the instance's master key. On S3, objects are always written with KMS encryption; on GCS they take the bucket's default encryption, so give the bucket a customer-managed key if you need one. Treat the bucket like the instance itself. ## Before you start * **A bucket.** Private. On S3, with KMS encryption; to restore in a different AWS account, use a customer-managed KMS key (`kms_key_id`) and grant the target account use of it, because the AWS-managed `aws/s3` key cannot be shared across accounts. On GCS, with uniform bucket-level access (the default for new buckets) and public access prevention; `kms_key_id` does not apply. * **Credentials for the source, GCS.** The exporter lists, reads, writes and deletes objects (it finds the previous snapshot, copies unchanged files from it and removes a failed attempt), so grant `roles/storage.objectAdmin` on the bucket; `roles/storage.objectCreator` alone makes snapshots fail. On GKE, use the pod's Workload Identity: annotate the chart's service account with `iam.gke.io/gcp-service-account` for a Google service account holding that role (or bind the Kubernetes service account directly). On Snowflake SPCS, create a GCP connection in Genesis with that service account's key, and name it in the request as `gcp_connection`. A restore only reads, so `roles/storage.objectViewer` is enough for the restoring cluster. * **Credentials for the source, S3.** These are AWS credentials. On EKS, the pod's identity (IRSA) with write access to the prefix. On other clusters (GKE, AKS), GKE Workload Identity and Azure identities grant nothing on AWS: either trust the cluster's OIDC issuer in AWS IAM and give the pod `AWS_ROLE_ARN` and a projected token at `AWS_WEB_IDENTITY_TOKEN_FILE`, or name a Genesis AWS connection in the request. On Snowflake SPCS there is no pod identity: create an AWS connection in Genesis with an access key that can write to the prefix, and name it in the request. * **Network egress from SPCS.** The Native App's external access integration must allow the bucket's endpoints: for S3, `.s3..amazonaws.com`; for GCS, `storage.googleapis.com` and `oauth2.googleapis.com`. * **The master key.** The target must use the same master key as the source, or it cannot decrypt any stored credential. Note the source's key before migrating; on SPCS it is the Snowflake secret behind `CORE.GET_ENCRYPTION_KEY()`. ## Take a snapshot Requires an administrator: config panel access, viewing agents, users and connections, and managing secrets, all connections and the scheduler, because the snapshot carries every stored secret and stops the server. Listing jobs needs only the viewing permissions. The `confirm` field must equal the instance's name, because the call pauses the instance. The destination must be an `s3://` or `gs://` prefix; local paths are accepted only by the `export-state` command inside the container. Name a Genesis connection for the destination's cloud: `aws_connection` for `s3://`, `gcp_connection` for `gs://`; a connection for the other cloud is refused. ```bash theme={null} curl -X POST https:///api/admin/state-snapshots \ -H "Authorization: Bearer " -H "Content-Type: application/json" \ -d '{ "destination": "s3://genesis-snapshots/prod/", "region": "us-west-2", "kms_key_id": "arn:aws:kms:us-west-2:111122223333:key/...", "aws_connection": "snapshots", "confirm": "" }' ``` Omit the connection on Kubernetes to use the pod's identity. The request first checks everything it can while the instance still runs (a write probe at the destination, the master key, and free space for the database copy); a failure is returned and nothing stops. A Snowflake-mode instance has no local database to measure, so its space check is skipped: if the scratch volume cannot hold the converted database, the export fails after the instance has stopped, and the instance then starts again. On success it returns `202` with a job id, then the instance stops. On a Helm chart install the container then ends and Kubernetes restarts it, so the startup probe, not the liveness probe, covers Genesis's boot; the pod's restart count goes up by one per snapshot. The chart's liveness probe allows at least 180 seconds of failures, enough for the server to stop and the exporter to start answering it. While the snapshot runs, the instance answers only its health checks and a progress endpoint: ```bash theme={null} curl https:///api/admin/state-snapshots/current ``` When it is back, list jobs and their outcome: ```bash theme={null} curl https:///api/admin/state-snapshots -H "Authorization: Bearer " ``` Each job reports its status (`succeeded`, `failed` or `aborted`), the snapshot id, its size, and any error. If the server has not finished shutting down two minutes after the request, it is stopped anyway and the snapshot proceeds. A container restart during a snapshot aborts that job; the instance comes back without it, and a new request starts over. ## Restore onto Kubernetes Set `restore.from` on a new install of the Genesis Helm chart. An init container fills the empty volume from the snapshot before Genesis first starts: The image must be a release after v1.19.93; earlier releases do not ship the restore. The chart refuses to render `restore.from` with a release tag (`vX.Y.Z`) at or below v1.19.93. A tag it cannot order, such as a build id, is not checked, and an image without the restore then fails in the init container. ```yaml theme={null} restore: from: "s3://genesis-snapshots/prod/" # newest complete snapshot # or a specific one: s3://genesis-snapshots/prod// # or on GCS: gs://genesis-snapshots/prod/ (region is S3-only) region: "us-west-2" secrets: existingSecretGenesisMasterKey: secretName: genesis-master-key # the SOURCE instance's key secretKey: GENESIS_MASTER_KEY serviceAccount: annotations: eks.amazonaws.com/role-arn: arn:aws:iam:::role/genesis-snapshot-reader ``` On GKE with a `gs://` snapshot, the restore reads the bucket through the pod's Workload Identity: annotate the service account with `iam.gke.io/gcp-service-account: @.iam.gserviceaccount.com` for a Google service account that can read it, and let that account be impersonated by `serviceAccount:.svc.id.goog[/]`. For an `s3://` snapshot the restore reads the bucket with the pod's AWS credentials, which the chart's init container takes from the standard AWS chain. The service account annotation above is the EKS way. On GKE or AKS the cluster identity grants nothing on AWS: trust the cluster's OIDC issuer in AWS IAM, mount a projected service-account token with that audience through `extraVolumes` and `extraVolumeMounts`, and set `AWS_ROLE_ARN` and `AWS_WEB_IDENTITY_TOKEN_FILE` in `config.extraEnv`. The init container receives the same environment and extra mounts as Genesis. The signed manifest identifies the master key used to take the snapshot. Genesis accepts a signature made with the current key or its bound previous key; the published development key is not trusted as a previous key. If `requires_previous_key_fingerprint` is present, secrets in the snapshot may still be sealed with the source's previous key, so that matching key must also be bound here as the current or previous key. Restoring a snapshot signed under the previous key restarts secret re-sealing; keep any previous key needed by the restored secrets bound until those secrets have been re-sealed. The restore refuses, and Genesis does not start, if: * the manifest signature does not verify under the current or bound previous key, or a key named by `requires_previous_key_fingerprint` is not bound here; * the snapshot comes from a newer Genesis release than the image (a development build, which has no release number, accepts a snapshot from any release); * the manifest lists a path outside the volume's trees, or the same path twice; * any file is missing, unexpected or does not match its checksum, or the database fails its integrity check. Files are downloaded into staging directories on the volume and verified there before being moved into place, so a failed restore leaves the volume as it was and removes its staging directories. Only files the signed manifest lists are downloaded. If the pod restarts while the verified trees are being moved into place, the next start finishes the move rather than restoring again. A volume that already holds state — any non-empty tree, or a file or symlink where a tree belongs — is never touched, so pod restarts and upgrades do not restore again. To restore over existing state, stop Genesis and run the restore by hand in a pod that mounts the volume: When durable SQLite is enabled, restoring into an empty volume leaves the existing Lakebase copies intact; they are restored over the snapshot's database files at the next startup. A forced restore replaces the Lakebase copies with the snapshot's durable databases. See [Durable SQLite stores on Databricks](/docs/deployments/Databricks/Genesis-On-Databricks#durable-sqlite-stores). ```bash theme={null} python3 -m scripts.uv.import_state_runner \ --from s3://genesis-snapshots/prod/ --region us-west-2 \ --target-root /genesis-data --runtime-root /app --force --yes ``` The restore also rewrites absolute file paths the source stored (uploaded files, SQLite connections, app registries) so they point where the chart mounts the volume (`--runtime-root /app`; without it, at the restored directories themselves). Only whole paths are rewritten: a directory that merely shares a prefix is left alone. The paths it rewrote, and any it could not, are recorded in `db/.snapshot-restored-from.json`, the one file the restore adds beside the snapshot's; later snapshots leave it out. Bots, the Slack app configuration and tunnel tokens belong to a runner, and a server serves only its own. The restore moves every runner-keyed row to the runner of the server that will serve the volume (`RUNNER_ID`, which the chart's init container shares with Genesis; `jl-local-runner` when unset). Without this, a snapshot from Snowflake SPCS, whose bots belong to `snowflake-1`, would restore bots the Kubernetes server never starts. The rows it moved are recorded in `db/.snapshot-restored-from.json` under `runner`. ## Migrating from Snowflake SPCS 1. Upgrade the Native App to a release with state snapshot support, and allow the bucket's endpoints in its external access integration. 2. In Genesis on SPCS, create a connection for the snapshot bucket: a GCP connection whose service account holds `roles/storage.objectAdmin` on a GCS bucket (the path to GKE), or an AWS connection that can list, read, write and delete under an S3 prefix. 3. Record the SPCS master key and store it as a Kubernetes Secret in the target cluster. 4. Take a snapshot with `gcp_connection` or `aws_connection` set. The exporter converts the Snowflake metadata tables into the Kubernetes SQLite database, reading every table as of the same instant through Time Travel. A table with no history at that instant (created after it, or kept without Time Travel retention) is read as it is at copy time instead. `reports/tables.json`, written beside the trees, lists every table, its row count, whether it was read at the shared instant (`point_in_time`), duplicate keys it collapsed (Snowflake does not enforce keys), and any column that exists on only one side. A row SQLite would reject for any other reason fails the snapshot rather than being dropped. The report is for the operator: it is not in the signed manifest, and a restore neither downloads nor checks it. 5. Install the Helm chart with `restore.from` pointing at the snapshot and the SPCS master key configured. 6. Configure authentication for the Kubernetes install (Auth0 or another OIDC provider). Snowflake sign-in does not carry over; users sign in again and roles are granted to their new identities. 7. Replace connections that relied on the SPCS service's own Snowflake session with regular Snowflake connections (key pair or OAuth). Before retiring SPCS, download `reports/tables.json` and review it. A table with `skipped_duplicates` above zero kept one row per key: where those rows differed in Snowflake, the other versions are not in the new database, so compare them in Snowflake and re-enter any that matter. A table with `point_in_time: false` was read later than the rest; check it if it changes often. Once the Kubernetes install is verified, suspend the SPCS service so both instances do not run against the same external systems.