Overview
Tool Name
Purpose
The data_connector_tools provide comprehensive connectivity and management for relational data systems. Set up secure connections, explore schemas, run parameterized SQL with safe limits, and pull detailed table information for documentation, modeling, or pipelines.Functions Available
_query_database: Execute SQL on a chosen connection with limits, params, and optional Google Sheets export._add_database_connection: Create or update a named connection with credentials, description, and access controls._delete_database_connection: Remove a connection that is no longer required._list_database_connections: List available connections with metadata for discovery and governance._search_metadata: Search databases, schemas, tables, and columns to locate relevant objects quickly._data_explorer: Browse objects by database and schema to refine scope and discover structures._get_full_table_details: Return detailed schema info for a table including columns, types, and constraints._run_sqlite_cli_commands: Run SQLite CLI commands, in SQLite’s safe mode, for local prototyping and inspection.plan_estate_comparison: Route a whole-database or whole-schema comparison through estate characterization and structural diffing._get_query_result: Read rows from a stored query result without running the SQL again._total_query_results: Add a numeric column across complete saved query results and store the totals as an embeddable table._assess_sql_complexity: Scan SQL for migration patterns and report a complexity rating without executing it.
Key Features
Execute SQL
Run parameterized queries with row limits, overrides when required, and optional export to Google Sheets.
Manage Connections
Add, list, and delete named connections with access controls for specific data agents or all data agents.
Search Metadata
Find databases, schemas, tables, and columns with fast keyword search to cut down exploration time.
Table Insights
Retrieve full table definitions including types, keys, and constraints for modeling and documentation.
SQLite Passthrough
Use SQLite CLI commands for quick local workflows like importing CSVs or inspecting schemas.
Assess SQL migration complexity
_assess_sql_complexity scans SQL for vendor-specific and procedural patterns and, when a
source dialect is supplied, attempts Databricks transpilation. It does not execute SQL or
read a database.
The result includes
overall_complexity, pattern findings, optional
sqlglot_transpilation, and parse_check (read, dialect, problems). The ratings are
SIMPLE, LOW, MEDIUM, HIGH, and UNKNOWN. When parsing fails and the pattern scan
would otherwise return SIMPLE or LOW, the rating is UNKNOWN, not an easy rating;
parse_check.problems explains why. The input may not be SQL at all, such as an ETL job
export. For bulk input, the result summary also counts UNKNOWN items.
Input Parameters for Each Function
_query_database
When a user asks about one environment, pass it as target_environment on every
_query_database call for that question, including follow-ups and schema lookups. A
successful result can include data_environment and data_environment_source whenever
Genesis can identify its source environment, whether or not a target was passed. The source
is declared for a matching connection declaration, names when inferred from names, and
ambiguous when the declaration and names disagree or names indicate multiple environments.
Account-wide listings such as SHOW DATABASES and SHOW ... IN ACCOUNT do not receive this
label. Label reported numbers with data_environment; a successful result can also include environment_note
when the connection or database names conflict with the requested target. Its source is
declared when the connection owner set its environment, or names when Genesis inferred
it from names; name-based evidence may be wrong. Do not present conflicting rows as the
requested environment’s answer. Omit target_environment when no single environment
applies to that call; for a comparison, pass the one environment each separate query reads.
Parameters
When reusing a match from
find_past_queries, check its connection_id and
environment before carrying the SQL into an environment-specific answer. The environment
is the connection’s current declared label, or null when it is undeclared or unreadable;
it may differ from the environment when the query originally ran.
Genesis refuses any SQL write that names its own ADMIN_AUDIT_LOG table;
reads remain allowed, and neither confirm nor an automated Flow can override
this protection. This keeps the administrative audit trail from being changed,
added to, or deleted through Genesis. See Admin audit log.
SQL that calls Snowflake’s SYSTEM$SEND_EMAIL function is refused before a
connection or SQL execution. This prevents email from bypassing review; send
messages through a connected mailbox with the Gmail tool instead.
For MongoDB JSON aggregation commands, a top-level $merge stage is treated as
a write and $out as destructive. These stages use the same confirmation and
Action Review gates as other governed writes; a connection declared
access_intent=read_only refuses them.
When SQL already exists in a saved repository file or upload, pass its reference
instead of retyping the statement. repo_id identifies the repository and
file_path is the path within it; optionally set commit to read that revision
instead of the current file. For example:
query and sql_file_ref cannot be supplied together. expected_sha256 requires
sql_file_ref; when omitted, the file is read as it exists at the selected
revision. The file must be non-empty UTF-8 text no larger than 1,000,000 bytes,
and it must not contain a Markdown code fence, leading sql tag, or USERQUERY::
prefix that this tool would strip. For saved-file calls, blank text and the
case-insensitive strings null, none, and undefined (with surrounding
whitespace) count as absent for query, sql_file_ref, and expected_sha256.
Any other query after the tool’s normal SQL-prefix and code-fence normalization is
refused as supplying both. The file text runs unmodified through the usual safety
gate, confirmation, Action Review, and permissions. The sql_file receipt always
identifies the file and resolved commit and includes the file’s
sha256. It includes executed_sql_sha256 and an executed-text preview only when
the response proves the statement reached the database: a successful result or the
database’s own rejection. A refusal before submission or a fault without proof has
no executed-text hash or preview. Their absence means execution is unproven, not
that the statement definitely did not run. Each call reads the file once, in
addition to the cost of running the query.
During a Prover spec-author run, _query_database binds reads on Snowflake and
Databricks to the catalog commissioned for that side: calls using the run’s
target_connection bind to target_catalog, and other calls use source_catalog
unless the source and target catalogs share one connection. For schema-based routing
and statements that read schemas unique to both sides, see the
author_spec catalog-binding guidance.
Two-part names therefore resolve in the catalog commissioned for that side rather
than the connection’s default. If the run’s catalog record cannot be read, the query
is refused instead of falling back to the connection default. A Snowflake read that
specifies role, and any statement that would take the governed write path, are
also refused when a commissioned catalog applies because those paths cannot bind
that catalog.
For a Databricks connection, use databricks_action(action="run_sql_statement")
when _query_database refuses a write for Action Review; it accepts one
statement per call. Otherwise, a Databricks write runs through _query_database
on its direct path. The engine check uses the registered connection type, not
only the connection id’s spelling. See Action Review
for when a warehouse write is held for review.
A native Databricks write is sent once. If the workspace answers with a redirect,
Genesis does not follow it or resend the statement: the result says the write was
delivered but its outcome is unknown. Do not retry blindly; inspect the target
first.
When a governed write is a multi-statement script, Genesis submits it as one
Action Review proposal. The card lists each parsed statement (or the whole text
when it cannot be split safely); because the changed tables cannot be determined
reliably, every rule that can cover the connection applies. A statement rejection
does not prove that earlier statements had no effect, so the result is reported as
uncertain. Built-in and Restricted Caller’s Rights Snowflake sessions that allow
one statement per call refuse multi-statement scripts before approval; send those
statements separately. A Snowflake scripting block is one statement and is listed
as a whole.
For Restricted Caller’s Rights role setup and session behavior, see Snowflake
connections.
Registered query transports refuse role because they cannot apply it.
The Databricks read-only gate recognizes DESCRIBE CATALOG [EXTENDED] <catalog>
and DESCRIBE VOLUME <name> as reads, so agents can inspect catalog details and
volume metadata through _query_database without a write confirmation.
For native Snowflake route refusals and the separate Snowpark session and Action
Review contract, see Snowflake Tools.
Do not retry a refused SQL write through Snowpark.
During a Prover spec-author seat run, reads use the connection access of the
person who commissioned the run; the seat does not need its own connection
grant. The seat remains read-only: data-connector write operations and exports
are refused, and its SQL reads must be a single read statement. Stored query
results are limited to results from the seat’s own thread, and a turn without a
person cannot read stored results. See Prover identity rules
for how the commissioning identity is established.
SQL response fields: see Connections.
When access to a connection is denied, the response has success: false and
error_code: "permission_denied", with an explicit refusal naming the connection. This is
distinct from credential_unavailable, which means Genesis could not obtain a credential.
Successful Snowflake SHOW object listings and queries that reference
INFORMATION_SCHEMA also return visibility_note; it explains that the result
only includes objects visible to the connection’s role. See Snowflake
integration guidance for
how to interpret that warning.
When a Databricks SQL Warehouse cancels a query at the deadline,
When a Snowflake query fails because the connection’s stored credential has expired (a programmatic access token past its expiry, Snowflake error 394401, or a session that expires again right after Genesis rebuilds it once), _query_database returns success: false with error_code: "credential_expired", an error worded for the connection’s sign-in method, and a credential_expired object naming the connection (connection_id, connection_type, connection_name, auth_type, personal_binding_used). An expired programmatic access token is not retried. In a conversation, Genesis also shows the person a card with the renewal steps; the agent is told so, and should not retry until the person says the credential is renewed.
_query_database returns timed_out: true with a timeout value and rewrite
guidance. For the spec-author and Adjudicator seats, the requested timeout is
capped at one minute less than the
estate_char.spec_author.no_progress_seconds stall bound (default 1800
seconds); other agents keep the requested timeout. See Advanced
Settings for the seat bound.
Inside a Databricks App, a Signed-in Databricks user (Databricks App)
connection runs _query_database as the authenticated user who sent the current
chat turn, using the user token forwarded by the App. The connection stores no
token, and queries are refused when there is no matching interactive chat user
or the user’s sign-in token is no longer valid. This identity mode is limited to
agent SQL calls in that user’s own chat session; flow and background work and
SQL exported to a repository do not run as that user. See Connect to Databricks
— Choose an authentication mode
for setup.
SQL API reads continue through the SQL API. For a non-READ statement, Genesis
refuses the request and points to _query_database when an Action Review rule
could govern either tool. If no rule could govern the write, Genesis runs it
synchronously through _query_database, applying that tool’s SQL checks,
confirmation, connection permissions and outcome handling. The answer has no
async statement handle. For _run_query_async(action="submit"), a successful
write reports completed_synchronously: true; a failure reports the attempted
write’s outcome. Supplying confirm does not bypass Action Review. This behavior
applies to Snowflake SQL API writes; Databricks follows the direct-or-reviewed
_query_database behavior above.
When a Snowflake SQL API write uses this path, its request database, schema,
warehouse and supported session parameters apply only to that execution. Invalid
or unsupported settings are refused before the write is sent; Genesis discards
the session afterward so those settings cannot affect later calls.
See Snowflake attribution.
For a write handled by Action Review, not_applied is used only when the
provider proves that the statement was refused before execution. Snowflake
compile-stage refusals are reported this way; timeouts, cancellations,
connection failures, and other post-submission errors remain uncertain and must
not be retried until the provider state is reconciled. An uncertain direct
warehouse write returns status: "reconciliation_required"; check the target
before retrying it.
When connection_id refers to a non-database estate with a registered query
transport, _query_database sends the normalized SQL through that transport instead
of the legacy DatabaseConnector. SQL safety checks and the usual max_rows,
max_rows_override, and max_result_bytes response bounds still apply. The call’s
timeout_minutes value is passed through as the transport’s per-request timeout;
the default is 5 minutes. Registered transports accept one statement and return
rows, so params and export_to_git_repo are not available on those connections;
retry without either option when the tool reports that limitation. This path is
available for connections such as palantir_foundry when their transport is
registered during startup.
For palantir_foundry.* connections, _query_database returns rows only; see the
Palantir Foundry guide
for its result-link limitation and when to use foundry_query for served_by.
If a registered query transport refuses a write for Action Review, use its
governed alternative; see Action Review
for when writes enter review.
For Iceberg worker-backed connections, read-only _query_database calls run on
their own connector lane, so a read does not wait behind an Iceberg bulk_copy
that may run for its full timeout. A query that may write an Iceberg table uses
the Iceberg lane instead and runs one at a time with Iceberg copies to protect
the table’s commit order.
Server-side cancellation is enabled by default for agent-facing
_query_database calls. Stopping an in-progress call asks the active database driver
to abort its statement on the database server, not only abandon the client’s
wait. This registration covers execution and result fetching, so a stop can
reach a query that is still returning rows. Drivers without a server-side
abort primitive refuse subsequent statements but cannot stop a statement that
is already running. In particular, SQL Server connections through pymssql
in a Databricks App cannot abort an executing statement; stopping the tool ends
the wait while SQL Server finishes that statement. Cancellation attempts are
bounded by
tools.cancellation.abort_deadline_seconds
and tools.cancellation.total_deadline_seconds.Reading result sets inline. For user queries, the LLM-visible
data array
follows max_rows (default 20) up to 500 rows inline; when the call has a
thread_id, full data stays available via query_id, and max_result_bytes
still caps payload size. Calls without a thread_id do not store a result or
return a query_id. When a preview is applied, the response also includes
preview_applied: true, preview_rows_returned, and preview_recovery with
recovery guidance; preview_note contains the same guidance for agents that
read the message text. To
read a whole result set back inline — e.g. DDL/GRANT statements generated
from information_schema — request a sufficient max_rows (add
max_rows_override=true when the connection caps the fetch, such as Snowflake
above 100 rows), or set preview_rows explicitly. Beyond 500 rows,
aggregate into one value (e.g. LISTAGG / array_join over
collect_list) or use export_to_git_repo=true and read the exported file
back. CSV exports preserve empty text separately from SQL NULL: empty text is
written as "", while NULL remains an unquoted empty field.Query result links. Successful stored results include query_id,
query_result_link, and a message line such as
[Title](query-result://<id>). To embed the result table inline in a
chat reply instead of a click-to-open link, append ?embed=1:
[Title](query-result://<id>?embed=1). In Slack, only an embedded link
posts a grid from a reply; a plain query-result:// link stays a link. Share
rows in Slack only when a person asked to see them or the table itself is the
answer they asked for. Otherwise, report your finding in words and use a plain
link if useful. Alternatively, set share_to_slack=true to post the grid
directly in the current Slack thread; it defaults to false and is ignored
outside Slack.Nested query values, including array or map cells containing decimals, are
normalized recursively for JSON serialization so they can be returned with
the query results.If
_query_database returns requires_confirmation: true, review the
returned classification and sql before retrying. Use confirm = true
only for blocked table-level destructive operations. Namespace-level
CATASTROPHIC operations require user_override to match the blocked
statement_type exactly, such as DROP DATABASE, and should be retried as
single-statement calls after explicit human approval.On the built-in local lakehouse connection iceberg.genesis_iceberg, Action
Review permits cleanup with one statement per call: use
DROP TABLE [IF EXISTS] namespace.table_name with confirm = true, or
DROP SCHEMA [IF EXISTS] namespace with user_override = 'DROP SCHEMA'
after the namespace is empty. Prefer IF EXISTS so a retry after a successful
drop is a no-op. When a remote Iceberg write is refused for Action Review,
use a Flow for the write; without that refusal it follows the direct path,
subject to the SQL safety checks. See Action Review
for when the review path applies.The built-in local lakehouse is unavailable to this tool in a Databricks App;
see connection mode chooser.When the optional SQL intent-match guard is enabled, _query_database can
also return requires_confirmation: true with severity: intent_mismatch
when the planned SQL does not clearly match the user’s latest request. In
that case, do not retry by only flipping confirm or user_override. Wait
for a fresh chat reply that explicitly names the exact operation and target,
then rerun with any confirmation fields that still apply.2. _add_database_connection
Parameters
allowed_bot_ids and allowed_user_ids apply only to legacy
CUST_DB_CONNECTIONS rows. New-style connections use connection scope and
Roles with Access instead. Agents cannot route this tool to legacy storage
or update a legacy row; use the Connections setup guide
for the boundary and the supported access-control path.
On success, the response reports owner_user_id and attributed_to_another from the
stored connection record. owner_user_id is the persisted owner, or
"unattributed" when the record names nobody; attributed_to_another is true when
the stored owner is somebody else, false when it is the caller, and null when no
owner is named. Use these fields rather than assuming the caller owns a connection.
3. _delete_database_connection
Parameters
A denied deletion is reported as not found. For the requester authorization rule,
see Connections — Agent context and access controls.
4. _list_database_connections
Lists legacy CUST_DB_CONNECTIONS rows and registry-backed connections for
types marked is_database or is_data_source. It can therefore include a data
source that is not a database connector, such as palantir_foundry, subject to
the caller’s connection access.
Parameters
Pass
fields when you only need a narrow connection inventory, for example
fields=["connection_id", "db_type"]. The access checks still run against the
full connection set, but masked rows contain only the requested keys. If neither
status nor health is requested, Genesis skips status construction and health
attachment; a masked response also omits dataset_connections_count.
Full rows include a health block with state (unmeasured, connected, or
error), last_checked_at (an ISO timestamp, or null when nothing was
measured), and basis. With the default include_health=false, the listing
does not probe connections. For a connection eligible in the caller’s context,
a query that returned success: true during this server process is reported as
connected; last_checked_at is the query’s success time, not a probe time.
Failed or refused queries do not update that reading, and a server restart clears
it. Otherwise the connection is unmeasured. The status text describes
configuration, not reachability. With
include_health=true, Genesis runs one SELECT 1 per probeable non-Iceberg
database connection, so the call makes one external round trip per connection.
Non-database data sources such as palantir_foundry remain unmeasured because
Genesis does not run a database probe for them. Connections that need
a per-user OAuth grant are left unmeasured rather than triggering an OAuth
prompt that would replace the rest of the listing. Local Iceberg catalogs are
opened while their status is built and therefore report connected or error
even when include_health is false; remote Iceberg catalogs remain unmeasured.
When an administrator has hidden the virtual System
Snowflake connection
(connection_id Snowflake or snowflake.Snowflake, owner System), it is
omitted from this list. Agents that still pass that id to _query_database
receive success: false, type: connection_disabled, and an error directing
them to use a configured Snowflake connection (for example snowflake.my_pat)
and re-run _list_database_connections(). Internal metadata, harvest, and
thread-storage callers that use the Snowflake connector directly are not
blocked.5. _search_metadata
Parameters
6. _data_explorer
Parameters
_search_metadata reflects harvested metadata for the connections available
in your deployment. With the Twin backend, _data_explorer(search_string=...)
searches Twin’s index and returns matching table hits also discoverable through
search_data_context;
non-table hits are omitted. When no indexed table matches, its response explains
that _data_explorer(connection_id=..., table=...) reads columns live from the
connection. The Snowflake sample schemas BASEBALL and FORMULA_1 are only
pre-seeded when an operator starts Genesis with INCLUDE_METADATA_SAMPLES=true.
In a person-attributed turn, metadata-search and browser results are filtered
to connections that person may see; the agent’s broader connection access does
not disclose another person’s connection metadata.7. _get_full_table_details
Parameters
8. _run_sqlite_cli_commands
Parameters
The SQLite command line always runs in SQLite’s safe mode. Commands that run
programs or reach files other than the connection’s database are refused:
.shell, .system, .load, .import, .read, .output, .once, .open,
.backup, .restore, .save, .cd, .log and .excel, plus ATTACH,
load_extension(), readfile() and writefile() in SQL. A refused command
fails the call with an error that says so. Load files into a table with
_load_to_sqlite; query results and .dump output come back in the tool’s
response.In a Databricks App,
_run_sqlite_cli_commands is refused: SQLite dot commands
can run shell commands as the App user outside the App’s confinement. Use
_query_database for SQL instead.Genesis prepends
.bail on, .dbconfig defensive on (SQLite’s defensive
mode: the schema table cannot be written directly with PRAGMA writable_schema
and the file’s raw pages cannot be written; ordinary DDL still works),
PRAGMA trusted_schema = OFF and PRAGMA busy_timeout = 30000 to every
SQLite CLI script, plus PRAGMA journal_mode = WAL on a read-write
connection; on a read-only connection the CLI opens the database read-only.
Before the CLI starts, a dot command outside the allowed list, SQL that
reaches other files (such as fsdir or zipfile) and control characters are
refused. The script also checks the database’s own schema before the first
command: any schema row (table, view, trigger or index) whose stored SQL mentions
zipfile or fsdir, the modules that read or write files outside the
database, stops the run with an error, and none of the commands run, even
when the word only appears in a name. The check runs once, at the start of
the run: a process outside Genesis’s SQLite connections that writes the schema
while the CLI runs is not covered. That includes the agent’s own code-execution
tool when it can write the database file: a database in a repository or worktree, or, while
durability is off, other paths the agent can name. On a root host that has the
sandbox user, the CLI runs as that user on a database in an agent repository or worktree (a
path under the repositories or worktrees directory), so such a writer gains no right it lacks.
On a root host without the sandbox user, a repository or worktree database is refused before the CLI starts; on a non-root host the CLI runs as the server’s user.
For any other file (a storage database, the rest of the Git
root, or while durability is off any other path an agent can name), it runs
as the server’s user. No run carries the server’s environment, and all start in
the root directory. The connection’s database file must be one an
agent may name: a file in the sqlite/ storage directory (ending .db, .sqlite or
.sqlite3), the workspace database, a file in demo_data/, a file in a repository, or, while durability is off, a
path outside Genesis’s storage (a configured worktree is accepted too, even under a temporary directory
that starts with genesis); Genesis’s own databases and caches are refused before
anything runs: that includes a name under the temporary directories that starts with genesis- or
genesis_ in any letter case, and a file in the storage root that has another hard link (a second
name of an internal database; hard-link backups of the storage tree on the same filesystem, such as
cp -al, therefore make every storage database refused until the extra links are removed). A connection string with a true uri flag and a file: URI is resolved to the file SQLite would open: its
filename is percent-decoded, its authority must be empty or lowercase localhost (anything else is refused),
mode=ro makes the CLI read-only, mode=memory or an empty or :memory: filename names no
file and is refused, and an opening parameter given twice or an encoded separator in the query is
refused. Without a true uri flag a file: name is an ordinary relative path, as SQLAlchemy opens it.
The CLI is launched on the resulting absolute path, so vfs, immutable and cache are not carried over. When a shared
database briefly reports database is locked, the tool waits and retries
once automatically. If the sqlite3 binary is missing on the host, the
response returns a clear tool error instead of an unhandled exception.9. plan_estate_comparison
Use plan_estate_comparison when a user wants to compare two entire database estates
or schemas, prove a migration matches, or see everything that changed between a
source and target. Do not loop _compare_query_results table by table; that tool
is limited to a single-query spot check.
For its bounded sub-query behavior, including when a sorted row bound is preserved,
see Connections — Query comparison safeguards.
plan_estate_comparison is also the entry point for proof-of-record questions such as
“is the migration verified,” “give me the current numbers,” “do the two sides
match,” “what is still open,” “where did those numbers come from,” or “has
anyone approved this, and when.” These questions are answered from the Prover’s
ledger, which records the verdict for each object, what was licensed, and who
signed it. The router is advertised on a fresh thread before other tool groups
are activated, so use it directly rather than asking the user to provide ids,
hashes, or API access.
Parameters
Both parameters default to
null. When supplied, each id is checked against the
connections this bot can reach. The response reports each side as verified,
not_found, unchecked, or not_supplied; an id that was looked up and not
found is replaced by a placeholder in the registration step instead of being
echoed as if it were known.
An id reported unchecked — its connection store did not answer, so nothing
confirms or denies it — ends the call instead: the response is
success: false with status: "connection_ids_unverified", an error naming
each unverifiable id and the reason its store gave, the same connections
report, and one recovery step (_list_database_connections, then call again).
No plan is issued, because every step of the plan would run on the id nothing
could confirm.
With no ids, the check reads nothing. With ids, it reads the legacy connection
table and the cached APP_CONFIG connection sections for the ids’ connection types.
The tool returns an ordered plan rather than running the comparison itself. First
activate the prover_tools group with
manage_context; then check prover_run(action="find_proofs") for an existing proof of record before
registering either estate. In a Genesis thread, register the source and target with
prover_estate(action="register", args={...}) and characterize each with
prover_estate(action="characterize", args={"estate_id": ...}); poll get_run
until each scan finishes, and run prover_run(action="prove", args={...}) with the two
returned snapshot ids. The
same estate and run verbs remain callable by name over MCP and HTTP; see the
Prover tool reference.
For BigQuery, pass the GCP project as database and the dataset as schema.
When find_proofs reports data.resolution.asserted: false, do not treat its empty
pairs as an unproved migration: inspect its candidates and call prover_run(action="find_proofs") again
with one candidate whose active value is true. The full resolution and candidate
field contract is documented in the find_proofs reference.
When both sides use the same database, keep their schema scopes distinct so the
comparison can identify the source and target correctly. Snapshot records expose
the selected schema names in schemas; null means the boundary could not be
resolved or recorded no schema selection, rather than an empty schema scope.
Snapshot ids are content-addressed. If the same id appears in more than one scan,
Genesis resolves it to one stable owner instead of choosing the newest scan: an
operator-registered estate takes precedence over the deterministic backfill estate
for that snapshot’s dialect:database group. Any other multi-owner case resolves to
the lowest estate id and is logged as ambiguous. A scan cannot bind a snapshot that
is already owned by a different estate; that scan closes with status: "error", leaves
the snapshot unbound for that scan, and reports the collision while preserving the
existing owner’s binding.
Measurement compatibility is checked automatically for mediated comparisons. Value
comparisons are admitted only when both snapshots carry the same known measurement
library version. If either version is unknown or the versions differ, value comparison
abstains with instrument_version_mismatch instead of reporting a match or difference;
object presence is still established. For recovery fields, use the
read_comparison response contract.
For pair-scoped digest re-measurement and comparable-column re-folding, including
the persisted quantized_remeasure and comparable_digest_refold disclosures, see
the estate API reference.
Comparison rows in the Estate workspace carry the server-resolved ruling_rollup
with settled person_ruled and instrument_void counts, plus open pair-wide
behavioral_open findings and the current Human Review review_decision when a
recorded decision is revalidated against the comparison’s population. For a legacy
comparison without a materialized population binding, Genesis rebuilds that
population from the run’s own evidence, so the comparison card and Human Review
resolve the decision’s standing the same way. The rollup also carries optional
review_readiness; when its population_complete is true and its
remaining_count is 0, a fully accounted run can read settled without a
recorded run decision. Legacy fallback is used only to measure the recorded
decision; it does not produce review_readiness, which remains a statement about a
materialized population. Featured comparison cards also expose
source_moved_since_approval and target_moved_since_approval in the rollup when
the batched movement read succeeds; these disclose that an estate CHANGED since the
reading its approval was recorded against, without withdrawing the approval. A re-scan
that measured the same content is not a change and the card stays silent. See the estate API reference
for the response fields and the lightweight findings projection used to resolve
them.
Use the estate API reference when consuming the
comparison-list highlight fields.
For historical comparison-event replays, use settled_at_run: its settled counts
describe the residue as it was when that run executed, rather than applying today’s
mutable rulings. A null value means the as-of-run record is unavailable, not that
the run had zero settled findings.
When reading those findings through prover_findings(action="read_comparison"), use each
row’s settlement
block and the comparison-wide settlement_counts to distinguish an observed difference
from one that is already accounted for; see the Prover tool reference.
The response has success: true, status: "estate_compare_plan",
activate_group: "prover_tools", connections, and the ordered next_steps array.
The optional optional_groups.source_contract_tools entry identifies the additional
group to activate for the optional rename-licensing step (propose_rename_correspondence).
The older
source_contract_tools comparison verbs still work for source-contract mining
and deviation triage, but prover_tools is the preferred
surface for comparing estates, grading a spec, and reading comparisons or findings.
The response also includes workflow_skill with skill_name: "prove-data-work",
an explanation of the optional spec-driven data verification playbook, and the
Genesis-thread activation call.
See the data-work proving skill
for the complete workflow guidance.
When this comparison feeds a reconciliation episode, use the estate API
reference for the episode response’s durable
finding #N and its cross-episode history; the comparison plan itself does not
allocate those issue numbers.
After the characterization and prove steps finish, the result
includes human_review_url pinned to the exact successful diff_hash. The
link is returned even when neither endpoint resolves to a registered estate;
in that case, Human Review admits the diff-only link and replay only from
persisted snapshot ownership, not from a database-name guess. When exactly one
endpoint resolves to an estate, replay_target is null; when both endpoints
resolve, it contains focus, vs, diff, and run, including two distinct
schema scopes within one estate. Use human_review_url to reopen the
comparison in Human Review. Present the result to the user as a link labelled
“See the proof” to proof_url, the comparison’s replay viewer (see
The proof page and Replay), and a decision they make as a
link labelled “Open the review” to human_review_url; never write the address or its hash
as text. A later prover_findings(action="read_comparison") call returns the same review
metadata for
the persisted diff_hash; if the hash is unknown, no review target is
returned.
In Human Review, approving a disposition-package item with promote_pi also
qualifies and ratifies its correspondence rule in that same signing action;
see the estate API reference for the
ratified_rules and refused_items response fields.
For renamed schemas or objects, pass schema_map or object_map to the
run_system_diff step for an ephemeral one-comparison addressing map. This
does not require rebuilding the target database or schema. To propose a durable
rename license for future diffs, use propose_rename_correspondence; it licenses
nothing until a human qualifies and ratifies the proposal in the Ratifications
worklist. Later comparisons report the renamed objects as licensed
object_relocated changes. The successful response also includes proposed_by, the
proposer recorded on the rule; it is not a caller-supplied parameter. Because rule
registration is idempotent by content-addressed rule id, redeclaring a rename that
already exists returns the existing row’s proposer rather than the latest caller.
See evaluate_estate_correspondence_rule
for the report-only fixture check before that handoff.
When a mapped diff is later graded for spec conformance, Genesis reuses the
diff’s recorded source-to-target object map to resolve source-side spec loci
against the target objects that the diff actually compared. Unmapped or identity
comparisons retain the existing grading behavior.
If you omit a map, Human Review links each unambiguous same-name
object_missing/object_surplus pair as a rename hypothesis with
cause="rename_hypothesis", so a reviewer can license it there.
When the target model declares a source table as input-only, because the build loads it
but carries it into no target object, do not add a schema-map entry just for that table.
The run_system_diff submission validates each map against both selected
snapshot inventories before creating a run. Schema-map keys and values must name
schemas on their respective sides; object-map keys and values must resolve to
objects on their respective sides. Qualified names are matched exactly, while a
bare object name is accepted only when it is unique on that side. An entry that
resolves to nothing or to more than one object returns 400 naming the entry
instead of producing a comparison whose mapped objects all appear missing or
surplus. An object_map value may be a one-element array, equivalent to its
single target name. An array naming several targets is rejected because the
addressing map is one-to-one; it is never reduced to one target. The submitted
object map is normalized to the schema-qualified names used by the diff.
To require a declared row identity for witness localization, pass row_identity
to the run_system_diff step as a JSON object such as
{"public.orders": ["order_id"]} (composite keys list columns in order). If a
key column was renamed, declare the two sides positionally, for example
{"public.orders": {"source": ["order_id"], "target": ["orderId"]}}.
The flat list is shorthand for identical names on both sides. When no single column
holds the key, a position may be {"coalesce": ["icn_adj", "icn_orig"]}; it uses
the first non-NULL value in that order. When an estate stores a missing value as exact
text, use {"nullif": ["icn_adj", ""]} as a key position, a coalesce operand, or a
concat part. For assembled keys, use concat; the full expression grammar and
derived-target row-key rules are documented in the
estate API reference.
Genesis probes each key on both sides before creating the run; a non-unique,
NULL-bearing, missing, or unresolvable key is rejected with 400 and never
silently falls back to identity-free digests. The verified key gates the pair
and localizes witnesses, while column comparisons remain order-independent
value-multiset digests. See the estate API reference
for the returned row_identity_basis and run-scoped row_identity_evidence.
The completed comparison summary also exposes licensed_breakdown so a licensed
total can be read by comparison and provenance, and object_measurement_keys so
Human Review can explain rename hypotheses supported by matching measured row
counts and boundaries. A categorical comparison whose value-count keys cannot be
aligned because float-rendering coverage is split or unknown is
indeterminate with cause="categorical_recipe_mismatch", rather than being
reported as an ordinary value-count change.
For model-aware comparison reading scope and not_exercised reconciliation, see the
Prover comparison guide
and the estate API response fields.
For the split-image row-count reconciliation rule, see the Prover’s target-mapping
guidance.
For append-only objects, a row-count shortfall on the later of two ordered reads is
reported with basis="append_only_shortfall_on_later_read" rather than treated as
observation skew. The comparison requires both read instants and their ordering; when
either is unavailable, it retains the normal abstention behavior.
When a ratified correspondence rule licenses a difference, the finding closes as
licensed_by_rule, not resolved_verified: the result means the difference is now
allowed, not that a repair was verified. The closing history retains the licensing
rule tags and the standing human diagnosis, and marks disposition_contested when
that diagnosis is target_defect. The run summary separates these closures in
closed_by_license; see the estate API reference
for the persisted response fields.
Completed comparisons also include consent_licensing, which reports the current
effective consent tier for rules that actually bound on that diff. Its auto,
checkbox, and ceremony entries each contain a count and the matching
rule_ids; basis is current_effective_tier. degraded_to_checkbox lists
shipped auto-tier rules held to one-click adoption because the dialect pair lacks
the required proven evidence. This is a current pack/evidence view, not an audit
of which operator action created a binding; estate-specific adoption is handled in
the estate API reference.
On SQL Server to Databricks, the standard type and rendering rules (R1, R2, R3, R9, R11,
R12, R13, R16, R17, R18, R19 and R20, with R20 applying in R15’s place) apply with no adoption. R18 and R19 bind only where the
target reports that UTC session time zone. They cover type names, renderings and missing key
declarations only: values are still compared, and a narrowing type still fails. A time column
stored as text is licensed only when reading both sides’ values proves it.
standard_rules lists the ones that bound with nobody deciding, each with the
assumption you can withdraw it by for an estate pair.
standard_rules_a_person_ratified lists the refs a person ratified for this estate
pair, which keep the licence that person gave them.
For a comparison of point-in-time snapshots or clones that were frozen with no
writes in flight, pass both_sides_static=true to the run_system_diff step.
This is an on-record attestation, not a tuning knob: the caller is recorded as
the attester, and the completed run summary includes quiescence_reliance with
the attester, the number of elements that relied on the attestation, the number
of those elements reported as unlicensed, and the number refused because an
object’s change stamp moved between reads. The declaration can turn an
observation-skew indeterminate result into unlicensed for covered mutable or
append-only content, but it is refused for objects that changed during the
comparison. Leave it at its default false for live databases that may change
between reads. Without the attestation, an indeterminate element remains
undecided: it does not close an open finding or count as verified resolution. See
the estate API reference for the
coverage_qualifier: "undecided" lifecycle field and finding-history behavior.
10. _get_query_result
Read a result that _query_database already stored, without running the SQL
again. Pass the query_id returned by the earlier query or pass its complete
query-result://<id> link. For example, _get_query_result(query_id="query-result://abc123")
returns the stored query, connection, columns, column types, rows, row counts,
title, and creation time. Only the first page of rows is retained; when the
response is truncated, re-run the query to retrieve rows that were not stored.
Each call reads the query-result store once.
Parameters
11. _total_query_results
Add a numeric column across results that have already been stored, without
rerunning their SQL or asking the model to add the values. Each input must have
a complete stored page; the tool refuses results whose rows may have been cut,
as well as failed results, prior totals tables, missing columns, and values that
cannot be added exactly. If one SQL query can calculate the aggregate, use
_query_database with SUM instead.
Parameters
Example:
total, count (values added), inputs, min, and
max. Integer totals stay integers; fractional totals are returned as exact
decimal strings. In a thread, the tool also stores a table with one row per
input and a final TOTAL row, and returns query_id, query_result_link, and
embed. Put the returned embed line in a reply to show the table inline.
Outside a thread, it returns the computed values without storing a table.
Each call reads the query-result store once per input and writes the new table;
it does not run a warehouse query.
Use Cases
- Cross database analytics Connect to Snowflake and Postgres, explore metadata, and run controlled SQL for KPIs with optional export to Google Sheets.
- Whole-estate migration checks
Call
plan_estate_comparisonto route a source-versus-target database or schema comparison to estate characterization and structural diffing. - Schema discovery
Use
_search_metadataand_data_explorerto identify the right tables and columns before writing joins. - Data documentation
Pull
_get_full_table_detailsfor core entities to build a lightweight data dictionary. - Prototyping with SQLite
Load CSVs with
_load_to_sqliteand test SQL logic via_run_sqlite_cli_commandsbefore promoting to a warehouse.
Workflow/How It Works
- Step 1: Add or verify a connection
Create with
_add_database_connectionor list with_list_database_connections. - Step 2: Explore and scope
Use
_search_metadataor_data_explorerto locate relevant objects. - Step 3: Inspect details
Call
_get_full_table_detailsfor types, keys, and constraints. - Step 4: Run SQL safely
Execute with
_query_database, setmax_rows, and preferparamsfor safety. Optionally export to Google Sheets. - Step 5: Maintain hygiene
Remove unused connections via
_delete_database_connectionand keep descriptions current.
Integration Relevance
- Snowflake workflows: Use alongside
snowflake_toolsfor staging and ingestion. - Harvesting: Feed
harvester_toolsto keep metadata fresh for search. - Collaboration: Export results with
google_drive_toolsand attach outputs viafile_manager_tools. - Project tracking: Link queries and docs in
project_manager_tools. - Versioning: Store reusable SQL with
git_action.
Configuration Details
- Provide valid SQLAlchemy URIs per engine and ensure network or VPN access is in place.
- Control access with the connection’s scope and Roles with Access, set through
connections_action(action="UPDATE", ...).allowed_bot_idscannot restrict a new connection — it applies to legacyCUST_DB_CONNECTIONSrows only, and_add_database_connectionrefuses any value but["*"]. - For engines that require it such as Postgres, provide
database_namewhen querying. - Result pagination and row limits help protect memory usage on large datasets.
Ensure the executing identity has the required database permissions for both metadata and data access. Missing grants will cause search or query failures.
Limitations or Notes
- Row limits
Default fetch limit is
20rows. For user queries, the LLM-visible preview followsmax_rowsup to500rows inline (preview_rowsoverrides when set explicitly). Exactly20rows may indicate more data. Increasemax_rows(andmax_rows_override=truewhen the connection caps fetches), setpreview_rows, paginate, aggregate, or export to git for larger sets. - Engine differences Some parameters and behaviors vary by engine. SQLite CLI passthrough is exclusive to SQLite.
- Performance and timeouts Long running queries can time out. Filter and aggregate early and avoid cross joins.
- Security Never include credentials in logs or outputs. Use parameterized queries to reduce injection risk.
- Exports Google Sheets export requires valid Drive credentials and permissions.
Output
- Connection management Success or error messages when adding, deleting, or listing connections.
- Query results
Rows with columns, row count, execution metadata, and optional Google Sheets export info. When
a query fails and Genesis can identify the provider’s code, the error response also includes
provider_error_code. A failed native Databricks SQL request appends the driver’s available HTTP, RPC, server, retry, timing, session, and query context toerror, so it remains visible in grade run records, and then carriesaccess_refused: whether the failure names missing access, judged on the driver’s message and the server’s own reason rather than on the HTTP status alone. - Metadata search Structured matches for databases, schemas, tables, and columns with relevance.
- Table details Column definitions, data types, keys, and constraints.
- SQLite CLI Raw command output and any error messages.

