> ## Documentation Index
> Fetch the complete documentation index at: https://docs.genesiscomputing.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Worked Example: A SQL Server to Databricks Migration
> One migration walked end to end — register and characterize both estates, compare them, author and grade a specification, settle the ambiguities, and record the review decision.
**One real migration, walked end to end.** Every number on this page was
measured on the run described here, not projected from what the product is
meant to do. The run finished with **1,290 elements matched, 243 differences
licensed, 3 elements the Prover declined to measure and that were approved as
exceptions, and nothing left open**, over a recorded review decision of
*approved with exceptions*.
Like the rest of this section, this walkthrough is human-facing: what was measured,
what came back, and what a person decided. The exact calls an agent makes at each
step are not restated here — they belong to the API reference for agents in the
Genesis repository under `docs/prover/`.
For the concepts these steps exercise, read
[Overview](/docs/Genesis-Prover/Overview) first. For what a run is and how to watch
one, read [How a run works](/docs/Genesis-Prover/How-A-Run-Works). For who may do
each step, read
[Roles and approvals](/docs/Genesis-Prover/Roles-And-Approvals).
## Scenario
WideWorldImporters is Microsoft's sample retail company: an operational SQL Server
database of **48 tables and 4,713,833 rows**, and a **14 table data warehouse** that
is rebuilt from it every night by **29 T-SQL stored procedures**.
In this run the warehouse was moved to Databricks. The Genesis agent Eve rebuilt all
14 warehouse tables as PySpark notebooks, working from a ticket and the legacy SQL
rather than from a translation of it, while the operational source was transported
as-is into a `wwi_bronze` layer so the rebuilt warehouse had the same raw inputs to
read.
What the reviewer needed at the end was not a green light from a row count. It was
proof that the rebuilt warehouse holds the same data as the original, table by table
and column by column, with every remaining difference either **licensed** as a known
platform difference between SQL Server and Databricks or **approved as an exception
by a named person**.
That is the bar. This run cleared it on the measurement — and **not on the
signature**: the exceptions and the decision were accepted under the instance's
service identity and journaled against a local reviewer identity, with a human
re-record still pending. [The review decision](#the-review-decision) sets out exactly
what that means and what a reproducer should do differently.
**Why a row count is not enough.** An earlier migration in this same package
reported zero failures with all 14 row counts matching exactly — while having
silently changed a customer-tier threshold from `>= 500000` to `> 100000`. No row
count and no schema diff can see that.
## Register and characterize both sides
**The step.** Each side is registered once, under a name, a role, a dialect, a
database and a schema list. The source was registered as *WWI DW — SQL Server
(source of truth)* against the `mssql.wwi_dw` connection, over the `Dimension`
and `Fact` schemas; the target the same shape against the Databricks catalog on
`databricks.wwi_gold`. **No credentials appear anywhere in the registration** —
the connection is referenced by name.
**What came back.** Two estate ids, which are the pair everything later hangs
off: `estate_44f6e19f936d4f56a1e5aa61b699121d` (source) and
`estate_fa46e5c1a4974e66841dec59effc8812` (target).
**What we decided.** The **pair** is the unit. A comparison is between two
characterizations of these two estates — not between two live databases.
**The step.** The source scan carried one **provenance assertion**: that the
`Lineage Key` column on 13 tables — `Dimension.City`, `Customer`, `Employee`,
`Payment Method`, `Stock Item`, `Supplier` and `Transaction Type`, and
`Fact.Movement`, `Order`, `Purchase`, `Sale`, `Stock Holding` and `Transaction` —
copies an engine-issued value from `Integration.Lineage`.
There is **no author field** in the assertion, and that is deliberate: Genesis
mints the asserter and the timestamp from the authenticated caller, so nobody can
assert on another person's behalf by putting their name in a payload.
**What came back.** The assertion is verified against the catalog when it is
used, so it cannot manufacture provenance the database denies. It is also part of
the **source boundary**, so adding it changes the pair fingerprint.
**What we decided.** Assert first, then scan, then bind, then compare. The
`Lineage Key` columns hold values issued by a sequence on `Integration.Lineage`,
but **no foreign key declares that relationship**, so nothing mechanical can reach
them and a name-pattern route is refused by design. A named person is the only
route.
**The step.** The same measurement against the Databricks side, in the target
role.
**What came back.** Preflight reported the estate **"not ready"** on two counts:
no ETL code assets attached, and no clone adapter for SQL Server.
**What we decided.** Proceed. Both of those concern the *behavioural stimulus*
path, not the characterize-and-compare path this run needed. **Read which light
failed before acting on a not-ready preflight.**
## Compare
**The step.** The comparison takes the two snapshots plus the vocabulary needed to
line the two engines up: an explicit case-insensitive object-name policy, a schema map
(`Dimension` to `dimension`, `Fact` to `fact`), and an object map for the four tables whose names
contain spaces — `Dimension.Payment Method`, `Dimension.Stock Item`,
`Dimension.Transaction Type` and `Fact.Stock Holding`. Both estates were declared
quiescent, with a note recording that no writers were active.
**What came back.** Omitting those maps produced **eight spurious object-level
findings** in one run, because Databricks lowercases identifiers and the four spaced
names were renamed.
**What we decided.** Always send both maps. And **re-run the comparison after every
binding change**, because a comparison consumes only the bindings that existed when
it ran.
## From 477 open findings to none
The first comparison, with nothing licensed, left **477 open findings**. Getting to
zero was not one act; it was nine, and only some of them were fixes to the migration.
Read every "open after" figure below as a count **read off a comparison that ran
after that step**. Neither a binding nor a fix moves a number that already exists —
the re-run is what moves it. That is the same rule as
[Bindings do not rewrite a run that already happened](#bindings-do-not-rewrite-a-run-that-already-happened),
and it applies to every row.
| Step | Open after | What moved it |
| - | - | - |
| First comparison, nothing licensed | 477 | baseline |
| Accept the platform defaults | 380 | roughly 100 closed by licence in one click |
| Build agent carries source nullability | 243 | 137 closed; the build agent's own defect |
| Build agent builds the lineage ledger | 233 | 10 closed; a gap in our ticket |
| Prover text-digest fix deployed | 198 | instrument defect, fixed not licensed |
| Remaining four Prover fixes deployed | 178 | instrument defects, fixed not licensed |
| Target primary keys, provenance assertion, re-bind, re-run | 77, 71, 65, then 14 | four steps in one session |
| R14 extended to the derived width, re-run | 1 | 13 `column_max_length` findings closed |
| Pack proposed, R15 bound, re-run | 0 | the spatial `column_type` finding closed |
| Abstentions approved, review decision recorded | 0 | the count does not move; the tile turns green |
The intermediate values **77, 71 and 65** are measured endpoints of a session in
which four steps landed together. Which step produced which drop was not recorded,
and is deliberately not attributed here rather than guessed at.
### Group findings by cause, not by table
The single most useful move on this run was sorting the 477 into five buckets by
*what caused them*:
Differences the two engines cause rather than the implementers. **License
these.**
The Prover measured it wrong. **Fix these — never license them**, or the check
goes blind.
Not part of what this transition undertook to reproduce.
The delivered warehouse is wrong. Fix the migration.
The fifth bucket is the one people miss: **downstream digests that cannot match while
a column digest is still unresolved.** They are not independent findings; they clear
when their cause clears.
**Fewer than half of the original 477 were defects in the migration.**
## The thirteen rules, and why two needed a ceremony
Correspondence rules are how platform variance gets licensed. **13 rules were bound
on this pair:**
| Tier | How it is bound | On this run |
| - | - | - |
| `checkbox` | One click of **"Accept platform defaults for this run"** binds them all | 11 rules |
| `ceremony` | **One at a time**, each with a deliberate sign-off | 2 rules |
The two ceremony-tier rules were **R14** (engine-issued surrogate keys renumber on
rebuild) and **R10** (text distinct-count under collation). They sit at that tier
because they license *value-derived* comparisons, which is a heavier thing to accept
than a difference in type spelling.
The pack for SQL Server to Databricks covers type spelling, unbounded text types,
integer and float widening, value rendering, unenforced key constraints, and
collation effects on distinct counts. The comparison's own pack-consent read listed
the pack's eleven rules:
| Rule | What it licenses | Tier |
| - | - | - |
| R1 | Engine type correspondence | `checkbox` |
| R2 | Unbounded text types to `string` | `checkbox` |
| R3 | Approximate-numeric and integer widening | `checkbox` |
| R4 | Value rendering — decimal identity, timestamp wall-clock | `checkbox` |
| R9 | Declared key constraints unenforced on the target | `checkbox` |
| R10 | Text distinct-count under collation | **`ceremony`** |
| R11 | Binary storage families | `checkbox` |
| R12 | Boolean rendering | `checkbox` |
| R13 | Temporal rendering — the licence covering timestamp precision at transport | `checkbox` |
| R14 | Engine-issued surrogate keys renumber on rebuild | **`ceremony`** |
| R15 | T-SQL spatial types to Spark `string` | `checkbox` |
**R15 arrived by upgrade and was bound last** — which is a story of its own, and it
is the second item under [What to expect](#what-to-expect).
## Rulings and abstentions
The last comparison left nothing open and **three indeterminate elements**. All three
were on `Dimension.City`, and all three trace to the same `geography` column:
* the boundary values of the `Dimension.City.City` column,
* the whole-table content digest on `Dimension.City`,
* the column digest on `Dimension.City.Location`.
**An indeterminate element is an abstention, not a finding.** The Prover is
declaring that it could not measure, rather than guessing. **No licence closes an
abstention** — a person approves it as an exception, bound to the run and to the
population it was measured on.
### Three abstentions, two packages
The three came back as **two** packages, not three, because the engine stamps two
distinct cause codes:
| Package cause | Members | What it covers |
| - | - | - |
| `rendering_excluded` | 2 | the whole-table content digest and the `Location` column digest — `measured_columns=[Location]` |
| `collation_uncorresponded` | 1 | the boundary values of the `City` column |
**Do not assume one object means one package.** The grouping is by *measured cause*,
and the count is whatever the engine measured. Approving both packages took the
unresolved count to 0 and unlocked **"Approve this run"**.
**Which route you get depends on your build, and this run predates the newer one.**
Approving abstentions as **packages** — one signature per measured cause — is
available **from Genesis commit `874f37c2ef` onward**. Before that, the fallback is
the **per-finding route**, once per abstention. **That is the route this run used**:
the same three abstentions were cleared one at a time, which is what took this run's
unresolved count to zero and made the review decision recordable. The two-package
grouping above is what the engine returns for these same three elements on a build
carrying that commit.
**Confirm the count actually dropped.** Both routes currently accept a stale
population hash and still return a success response, while the read side ignores
the approval. This is a known open defect. Check that the unresolved count moved
rather than trusting the success response.
## The review decision
The run was recorded as **`approved_with_exceptions`** on run
`ecrun_bd4c0f2ea92141aebaa113a0a1e8ce5e`, over diff
`56680664486b74c6ecbfe12452bcf45f37e83498eb0c98c549a823838d680d00`.
| Final count | |
| - | - |
| Matched | 1,290 |
| Licensed | 243 |
| Indeterminate (approved as exceptions) | 3 |
| **Open** | **0** |
The unresolved count must be zero before a decision can be recorded at all, which is
why the abstentions come first. And the relationship worth internalising runs the
other way from most dashboards: **zero open findings only makes the decision
available to record. The tile turns green off the decision, never off the count.**
**Who signed this one, stated honestly.** Both the exception writes and the decision
write on this run were accepted under the instance's **service identity** and are
journaled against the local reviewer identity — **not against a person**. A Genesis
fix requiring a genuine human login on those two routes was in flight at the time of
the run, and **a human re-record was pending**. Anyone reproducing this should drive
the last step from the Prover interface while signed in, so the decision carries
their own name.
[Roles and approvals](/docs/Genesis-Prover/Roles-And-Approvals#on-a-stack-with-authentication-turned-off)
explains why the local attribution is deliberately self-describing.
## Author and grade the specification
Everything above works with no specification attached. This run also exercised the
other half: the **spec agent** authoring the Prover specification directly against
the SQL Server source, reading the estate rather than a description of it.
Three tickets were sealed on 2026-09-07:
| Ticket | Structural items | Obligations | Settlements | Finalize rounds |
| - | - | - | - | - |
| `Dimension.Date` | 15 | 15 | 5 | 1 |
| Six fact surrogate keys | 12 | 36 | 2 | 2 |
| `Fact.Stock Holding` | 10 | 11 | 3 | 2 |
Their sealed specification names are `wwi_mig_1a_dimension_date`,
`wwi_mig_1b_surrogate_keys` and `wwi_mig_1c_stock_holding`.
**Every asserted number was re-derived against the estate: 51 of 51 source-side SQL
statements clean, zero misbound loci, casing exact.**
Two of the agent's judgment calls are worth reading, because they are the kind of
thing a specification usually gets wrong quietly:
On the surrogate keys the author read the ticket's *"excluded"* as **values not
compared, columns still required** — and backed that by measuring
`WWI Invoice ID` at **70,510 distinct over 228,265 `Fact.Sale` rows**, against a
composite of 228,265.
On `Fact.Stock Holding` it encoded the no-time-predicate
`ORDER BY [Valid To] DESC` resolution, put **none** of the sibling window
operators in any obligation, and **proved the rival reading wrong across 227 of
227 rows**.
### Three caveats measured on this run
The naming gate refuses a locus that packs a column into `schema.table.column`,
reading it as catalog-qualified. Use the separate `column` field. This cost a
finalize round.
A dropped connection is reported as a *specification* defect, with wording that
reads like the specification is wrong. Retry before changing anything.
Unaliased duplicate result columns collapse silently to the first value, and
nothing warns you.
## What to expect
Four things on this run surprised people who already understood the product. All four
are the system working; none is a bug you can wait out.
### A provenance assertion changes the source boundary
The assertion rides on the source characterization, and adding it **changes the pair
fingerprint**. That starts a new estate pair, and **every rule binding must be redone
on it by a person**. Bindings cannot be inherited from yesterday's pair. Order the
work *assert, scan, bind, compare*, and you pay this once instead of twice.
### A new pack rule version is not offered until the pack is proposed again
A new rule version has **no registry row on your pair** until the pack is proposed
again. Until then the Correspondence tab reads *"Nothing is waiting for your sign-off"*
while the finding that rule would license stays open — and **nothing in the interface
hints that a rule exists but was never offered**. This cost twenty minutes on this run.
When an already-shipped rule's definition changes, its earlier content ID remains
recognized as shipped. An unratified copy under that earlier ID therefore remains
identified as a platform default in unscoped rule worklists; the newly minted ID is a
separate version and still needs to be proposed for your pair.
Make it a reflex: after any Prover upgrade, ask what the pack ships against what your
pair has been offered, and propose the pack if anything is unproposed. On this run,
immediately after the upgrade that shipped the spatial rule, the reading was
**11 shipped, 10 proposed, 1 unproposed**.
### Bindings do not rewrite a run that already happened
Immediately after signing, the page showed **0 licenses and the same findings as
before**. Nothing is broken: a comparison consumes the bindings that existed **when it
ran**. Re-run the comparison with the same inputs — and **quote that diff hash**, not
the one you were looking at while you were signing.
### Proposals from another estate may appear on older builds, and refuse to bind
On builds predating the queue-hygiene fix, the Correspondence queue can show proposals
drawn on objects that are **not in your database**. On this run four appeared, against
`dbo.PRICING.IWGHT` and `dbo.CLOSE.IWGHT` — objects belonging to an unrelated estate.
They **refuse to bind**, which is the right outcome. But they clutter the panel, and
**approving one records your name against another project's licensing**. Check object
names before signing, and sign only what is tagged *Platform default*. The fix merged
and deployed on 2026-09-06, and the four are gone from this queue.
## The artifacts you can cite
Every result on this page has an address. The deep links below are shown as paths
against your own Genesis host; on this run they pointed at an internal lab instance
and required sign-in. Treat them as a map of what to wire up.
| Artifact | Where |
| - | - |
| The comparison (deep link) | `/estate/?view=review&focus=estate_44f6e19f936d4f56a1e5aa61b699121d&vs=estate_fa46e5c1a4974e66841dec59effc8812` |
| Migration portal (deep link) | `/apps/?kind=apps&appId=wwi-migration-portal` — the Apps panel's own deep link, category *Demos* |
| Durable board, served by the product | `/api/git/wwi_migration/boards/wwi_migration_board.html` (sign-in required) |
| Final diff hash | `56680664486b74c6ecbfe12452bcf45f37e83498eb0c98c549a823838d680d00` |
| Final run id | `ecrun_bd4c0f2ea92141aebaa113a0a1e8ce5e` |
| Build agent thread | `litellm_thread_5ae36a14-0843-4344-a63c-5cbd59815a26` (16 messages) |
| Build agent access-check thread | `litellm_thread_f50e96a0-5da4-464f-bea5-617b385b6568` (4 messages) |
| Thread exports | `wwi_sqlserver_to_databricks/SOLUTION/our_results/threads/` |
| Databricks orchestrator run | `253387974231014` — the run the approved tables came from |
| Databricks stock-holding re-run | `669533985326312`, job `293145873178136`, notebook `14_fact_stock_holding` |
| Walkthrough package | `wwi_sqlserver_to_databricks/` in `genesis-computing-ai/customer-pocs` |
## The builds these measurements were taken on
| Component | Reference | Status |
| - | - | - |
| Instance image, final green comparison | `3c34f44a57ba9a8a5c7c7061b002f400403b4ed7`, deployed 2026-09-06 21:53Z | The final comparison ran 2026-09-06 22:34Z on **this** image, as every artifact in the package records. `66534d541f…` was deployed later that night (2026-09-07 02:18Z) and carries the human-authority guard on review decisions |
| Package-approval fix, deployed later | `874f37c2ef9c2ca64c4920dfee705a5055d9ff2e` (2026-09-07) | Artifacts carry the short form `874f37c2ef` |
| Spec-agent runs | `30970012e9` and `7d0e1b3414` | `30970012e9` (tagged `acceptance-head/2026-09-07a`) ran the `Dimension.Date` ticket; `7d0e1b3414` ran the other two. Both carried the T-SQL classifier fix — merged to main as `b18899ff23` — as a local merge, and the sealed seals stamp the local merge commits `a73a19b6a3` and `f45416ce0a` |
**All of the above are superseded by current `main`.** They are stated as the builds
these measurements were taken on, not as versions to deploy.
**Deploy note carried from the run.** Use the exact 40-hex commit that
`git rev-parse` resolves, and confirm the registry tag exists before setting the
image. A hand-completed commit took the instance down for **12 minutes**.
## Acceptance of record
The spec agent's acceptance-of-record re-run **has now happened, on `main` at
`049189d10880547fb1520777672c9f530bfa22fb`** (`genesis_version` `dev.049189d108`). The
three tickets were run one after another, on a stack whose model was set by APP\_CONFIG
alone — Bedrock, `global.anthropic.claude-opus-5` — against the `mssql.wwi_dw`
connection registered in-process, under constitution `constitution_v8_12.md`
(sha256 `73da08631eb1ae08bcea94c4858778b199d443ecdb5e831014dd187ac2c4a45a`).
**All three sealed.** The numbers below are that run's; the table under
[Author and grade the specification](#author-and-grade-the-specification) records the
earlier pass, on the spec-agent builds `30970012e9` and `7d0e1b3414`.
| Ticket | Sealed specification | Structural | Obligations | Settlements | Finalize rounds | Wall |
| - | - | - | - | - | - | - |
| `Dimension.Date` | `wwi_mig_1a_dimension_date` | 15 | 12 | 3 | 1 | 26m15s |
| Six fact surrogate keys | `wwi_mig_1b_surrogate_keys` | 18 | 30 | 3 | 2 | 39m57s |
| `Fact.Stock Holding` | `wwi_mig_1c_stock_holding` | 11 | 10 | 3 | 1 | 50m39s |
Each sealed specification has a hash, and the hash is what a later reader cites:
| Sealed specification | `spec_hash` |
| - | - |
| `wwi_mig_1a_dimension_date` | `78d5f4c34fffd512724b57f52d677311cd09672bab521ec47986b37a13d14408` |
| `wwi_mig_1b_surrogate_keys` | `a56f8814cf48e67c0eb1927e625f83d6f31729db64a9d9fb6c7fa2ef794d40f9` |
| `wwi_mig_1c_stock_holding` | `25261870351645ad07a00cd04ca42f56ef80746f481827b975bcce27c19404ec` |
**The one second round is the part worth reading.** The surrogate-keys ticket's first
finalize was **refused on substance** — the specification carried no membership
obligation at all. The author added `O25` through `O30` and sealed on the second round.
That is the gate doing the job it exists for: a specification that measures nothing
about membership does not become a record just because it is well formed.
Two of the three wall times read straight. The third does not: **about 29 of
`Fact.Stock Holding`'s 50 minutes were relay-drop backoff, not authoring.**
### What the run was read against
The readings were taken against a criterion **pre-registered before the run**, so the
bar was not moved to fit what came back.
The surrogate-keys ticket again read the ticket's *"excluded"* as **columns
required, values not compared**, and measured **228,265 / 70,510 / 228,265** — the
`Fact.Sale` row count, the distinct `WWI Invoice ID`, and the composite. Its
invoice-line natural key is `(WWI Invoice ID, Description)`, **exactly unique**.
`Fact.Stock Holding` encoded `ORDER BY [Valid To] DESC` with **plain equality** and
an **Unknown fallback of 0**, and put **no sibling window operator in any obligation
SQL**.
Every asserted number re-derived against the estate. The grader counts **by
`target_sql` key, not by obligation**, so its denominator is not the obligation
column above and is not meant to match it. It returned **13/0** on
`Dimension.Date`, **36/0** on the surrogate keys and **11/0** on
`Fact.Stock Holding` — **nothing failed on any of the three**.
**44 loci, zero misbound.** Every one is schema-qualified and **none is
database-qualified**. Casing is exact.
All three carry complete provenance: **surface seat, bot `spec-author`**.
The classifier **tripped zero times** across the three tickets, and **nothing was
refused by name**.
The first of the [three caveats](#three-caveats-measured-on-this-run) above no longer
bites: the three-part column loci that cost a finalize round the first time were
**accepted first time** on this run, and **finding 34 is closed.**
**Seen on this build, and not part of the acceptance.** Transport drops (`08S01`)
were returned to the author as *specification* defects (finding 40). Relation and
column names that need quoting are now rendered with the registered connection's
dialect, so a name containing a space is no longer probed as two SQL identifiers.
## Next
* [Overview](/docs/Genesis-Prover/Overview) — what the Prover measures and how its six
components fit together.
* [How a run works](/docs/Genesis-Prover/How-A-Run-Works) — statuses, provenance, and
where to watch.
* [Roles and approvals](/docs/Genesis-Prover/Roles-And-Approvals) — what lands in
front of you and what each button attests to.
**For agents and integrators.** The exact calls, request shapes and refusals behind
every step above live with the code, in the Genesis repository under `docs/prover/`.
This page is the account of one run and deliberately does not restate them.