> ## Documentation Index
> Fetch the complete documentation index at: https://docs.genesiscomputing.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Databricks Action Tools
> Manage Databricks notebooks, workspace files, SQL execution, and jobs from one action-driven Databricks tool.
## **Overview**
### **Tool Name**
```json theme={null}
databricks_action
```
### **Purpose**
The **databricks\_action** tool provides programmatic control of Databricks workspaces. It enables notebook CRUD, cross-workspace copy and move for existing Workspace notebooks and files, workspace file sync and upload flows, SQL execution, permissions management, and job or cluster interactions so you can orchestrate Databricks work without using the UI.
#### **Functions Available**
1. **`databricks_action`**: *Performs Databricks operations, including notebook lifecycle actions, workspace file sync, SQL execution, job control, and status checks through the Databricks APIs.*
### Key Features
Create, read, update, list, and delete notebooks programmatically in workspace folders.
Run notebooks on clusters with `params` and monitor run state, logs, and results.
Sync repo-backed notebooks and workspace files into Databricks Workspace, export notebook content in supported formats, or copy and move existing Workspace notebooks and files between configured Databricks connections.
Manage access to notebooks and align with workspace sharing policies.
Target specific clusters or jobs for scheduled flows, and submit single-statement or batched SQL work.
## **Input Parameters for Each Function**
### **`databricks_action`**
**Parameters**
| **Name** | **Definition** | **Format** |
| - | - | :-: |
| **action** | Databricks operation to perform. Common values include `copy_workspace_object`, `move_workspace_object`, `edit_workspace_file`, `create_notebook`, `list_notebooks`, `list_clusters`, `run_notebook`, `run_notebook_live`, `run_notebook_cells`, `upload_file`, `sync_from_git`, `run_sql_statement`, `run_sql_batch`, `list_metric_views`, `register_uc_assets`, `describe_uc_registration`, `apply_uc_comments`, `apply_uc_tags`, `declare_uc_keys`, `create_metric_view`, `create_domain`, `list_domains`, `assign_to_domain`, `list_tag_policies`, `create_tag_policy`, `create_uc_page`, `bulk_create_uc_pages`, `publish_uc_page`, `list_uc_pages`, `delete_uc_page`, `publish_context_subgraph`, `publish_external_metadata`, `list_external_metadata`, `delete_external_metadata`, `publish_external_lineage`, `list_external_lineage`, `delete_external_lineage`, `delete_pipeline`, `create_folder`, `delete_folder`, `describe_designer_templates`, `generate_designer_flow`, `render_pipeline_as_designer_flow`, `publish_designer_flow`, `list_designer_files`, `get_designer_flow`, `delete_designer_flow`, `create_job`, `update_job`, and `run_job`. | String (required) |
| **path** | Workspace path for notebooks, folders, or workspace transfer source objects. For `copy_workspace_object` and `move_workspace_object`, this is the absolute source Workspace path. For file actions, also accepts DBFS or Unity Catalog paths such as `/Volumes/catalog/schema/volume/file.csv` or `/FileStore/shared_uploads/data.csv`; `delete_file` uses the Unity Catalog Files API for `/Volumes/...` paths and the DBFS API otherwise. | String |
| **target\_connection\_id** | Destination Databricks connection for `copy_workspace_object` or `move_workspace_object`. The `connection_id` parameter selects the source workspace. | String |
| **target\_path** | Absolute destination Workspace path for `copy_workspace_object` or `move_workspace_object`. Must begin with `/` and must not end with `/`. | String |
| **overwrite** | Whether to overwrite an existing notebook or Workspace object. Transfer actions default to `False` and also require `confirm_overwrite=True` after explicit user approval. | Boolean |
| **confirm\_overwrite** | Set to `true` only after the user explicitly approves replacing an existing destination object. Never self-confirm. Default: `False`. | Boolean |
| **confirm\_move** | Set to `true` only after the user explicitly approves deleting the source after a verified copy. Never self-confirm. Default: `False`. | Boolean |
| **content** | Notebook source or file content for create or import style actions. | String |
| **language** | Notebook language: `PYTHON`, `SCALA`, or `SQL`. For `execute_command`, use `PYTHON`, `SCALA`, `SQL`, or `R`. | String |
| **command** | Command text for `execute_command` or `execute_command_in_context`. | String |
| **command\_id** | ID returned by an asynchronous command; use with `context_id` and `cluster_id` for `get_command_status` or `cancel_command`. | String |
| **context\_id** | Execution context ID returned by `create_execution_context` or an asynchronous command. Used by `execute_command_in_context`, `get_command_status`, `cancel_command`, and `destroy_execution_context`. | String |
| **wait\_for\_result** | For `execute_command` and `execute_command_in_context`, wait for a terminal result. Default: `True`. Set to `False` to submit without polling and use the returned `command_id` and `context_id` for follow-up. `execute_command` with `language="SQL"` requires `True`. | Boolean |
| **format** | Import or export format. Default: `SOURCE`. | String |
| **connection\_id** | Databricks connection id. This is the preferred authentication input. You can also provide it as `params.connection_id`; if both locations contain a non-empty id, they must agree. The selected connection supplies the host, credentials, and (for SQL) warehouse. If omitted in both locations, the tool falls back to `DATABRICKS_HOST` and `DATABRICKS_TOKEN`; that environment-token fallback is refused when Genesis runs as a Databricks App. | String |
| **params.catalog** | Unity Catalog catalog to inspect with `list_metric_views`. | String (required for `list_metric_views`) |
| **params.schema** | Unity Catalog schema to inspect with `list_metric_views`. The action lists only this schema. | String (required for `list_metric_views`) |
| **params.pipeline\_id** | Lakeflow Declarative Pipeline ID for `delete_pipeline`. | String (required for `delete_pipeline`) |
| **params.cascade** | Whether `delete_pipeline` also requests deletion of datasets published by the pipeline. Required for `delete_pipeline`; pass `true` or `false` explicitly. | Boolean (required for `delete_pipeline`) |
| **params.warehouse\_id** | SQL warehouse used by `delete_pipeline` to check internal materialization tables in `information_schema.tables`. Optional; an explicit ID takes precedence over the warehouse metadata stored with the connection. The warehouse must already be `RUNNING`; this action does not start it. | String (optional for `delete_pipeline`) |
| **repo\_id** | Git repository id used with `sync_from_git`, `create_notebook(file_path=...)`, `upload_file`, or `download_file(file_path=...)`. Default: `default`; required when `file_path` is used. | String |
| **file\_path** | Repository-relative path for `upload_file`, the source for `create_notebook`, or the destination for `download_file`. `repo_id` is required; absolute paths and paths naming or resolving into `.git` are refused. | String |
| **source\_paths** | Array of file or directory paths in the Git repository to sync with `sync_from_git`. | Array\[String] |
| **target\_workspace\_path** | Base Databricks workspace path for `sync_from_git`. | String |
| **preserve\_structure** | Place each selected source directory under `target_workspace_path` by its basename while preserving descendants; a single source file lands by basename. Set to `False` to flatten every file to its basename. Default: `True`. | Boolean |
| **validate\_transfer** | Validate transferred files after `sync_from_git`. Default: `True`. | Boolean |
| **validation\_mode** | Validation mode for `sync_from_git`: `python`, `databricks_notebook`, `auto`, or `skip`. Default: `auto`. | String |
| **auto\_create\_directories** | Automatically create missing parent directories in the Databricks workspace during `sync_from_git`, `copy_workspace_object`, or `move_workspace_object`. Default: `True`. | Boolean |
| **params** | Action-specific payload. For `edit_workspace_file`, provide `old_text`/`new_text` or notebook cell-edit fields; optionally guard with `expected_sha256` and preview with `dry_run`. For `run_sql_batch`, provide `params.statements` as an array of up to 50 SQL strings. For `run_notebook`, use it for base parameters. For `run_sql_statement`, provide a single SQL statement plus optional warehouse or polling settings. For `run_job`, provide optional `job_parameters` as an object for Databricks Jobs API 2.2 job parameters and optional `idempotency_token` to prevent duplicate submissions. | Object |
| **params.confirm** | For command actions, set to `True` only after the user approves running code that may write data. | Boolean |
| **verbose\_errors** | For `check_run_status` and `run_notebook_live`, return the raw Databricks error payload instead of the trimmed `error_digest`. This can add 100+ KB of context; use it only when the digest and spilled trace are insufficient. Default: `False`. | Boolean |
**Genesis Tip**\
When running production jobs, pass `params` for date ranges or environment flags instead of hardcoding them in notebook cells.
#### **Action Routing**
| **Goal** | **Recommended action** | **Notes** |
| - | - | - |
| Copy or move one existing Workspace notebook or file between Databricks connections | `copy_workspace_object` or `move_workspace_object` | Supports `NOTEBOOK` objects in `PYTHON`, `SQL`, `SCALA`, or `R`, and `FILE` objects. `connection_id`/`path` select the source; `target_connection_id`/`target_path` select the destination. If only workspace names are known, call `list_connections` first. Transfers are verified by exact content hash without returning object content to the agent. |
| Edit part of an existing Workspace notebook or file in place | `edit_workspace_file` | Replace one unique exact text span in a `NOTEBOOK` or `FILE`, or change one confirmed cell in a notebook. Workspace content stays server-side; use `dry_run=True` to preview the bounded diff without writing. |
| Sync repo-backed notebooks or workspace files into Databricks Workspace | `sync_from_git` | Supports notebook source extensions `.py`, `.sql`, and `.scala`, plus workspace file extensions `.md`, `.csv`, `.json`, `.yaml`, `.yml`, `.txt`, `.toml`, `.cfg`, `.ini`, `.xml`, `.html`, and `.r`. Notebook-source files are imported as notebooks and lose their source extension in the workspace. |
| Create or update a single workspace notebook | `create_notebook` | Use this for one notebook at a known workspace path. For one notebook from Git, you can also use `create_notebook(from_git=...)`. |
| Upload a file to Unity Catalog Volumes or DBFS | `upload_file` | Use `/Volumes/...`, `/FileStore/...`, or `/dbfs/...` destinations only. `upload_file` does not support `/Workspace/...` or `/Users/...` paths. |
| Stage S3 objects into a Databricks Volume or DBFS path | `stage_from_s3` | Requires `params.source_s3_uri` (or `file_path`) and `path` (or `params.target_path`). Supports optional `params.aws_credential_name` / `params.aws_connection`, `params.aws_region`, prefix filtering, and volume auto-create options. |
| Run one SQL statement with status polling support | `run_sql_statement` | Best when you need one statement id, explicit polling, or SQL Statement API specific options. |
| Run multiple SQL statements in order | `run_sql_batch` | Requires `params.statements` and accepts up to 50 SQL strings in one call. |
| Delete a Lakeflow Declarative Pipeline | `delete_pipeline` | Requires `pipeline_id` and an explicit `params.cascade`. With `cascade=true`, the action inventories metric views and internal materialization tables in the pipeline's target schema before deletion; inspect the returned survivors before dropping anything. |
| Update an existing Job definition | `update_job` | Use `params.mode='replace'` to overwrite the submitted task collection or `params.mode='merge'` to merge submitted tasks by `task_key`; the action reads the job back and reports whether the checked fields persisted. |
| Trigger an existing Job with named job parameters | `run_job` | Requires `job_id`. Pass `params.job_parameters` as an object to use the Jobs API 2.2 contract; `params.idempotency_token` is forwarded when supplied. Without a mapping for `job_parameters`, existing parameter styles use the Jobs API 2.1 endpoint. The response returns `run_id`; use `check_run_status` to monitor it. |
| Inspect workspace clusters before selecting cluster-based compute | `list_clusters` | Uses `connection_id` and returns active, terminated, and other clusters. This does not list serverless compute or SQL warehouses, so an empty list is not an inventory of all available compute. |
#### **In-thread completion notices**
When a Databricks action leaves a job run, pipeline update, or SQL statement running in the current Genesis conversation, Genesis can watch it and send the terminal result to that thread. If the server restarts while the completion notice is only queued in memory, Genesis retains the watch and retries delivery. Follow the tool response's status-check instructions if it does not confirm that a watch was set up.
For a signed-in person, `run_notebook` also returns `project_tracking` when a live migration project published that notebook on the same Databricks connection. Genesis attempts to file the submitted run under that project; the field says whether it was tracked or why filing failed. See [checked project records](/docs/Genbot-Toolkit/Project-Manager-Tools#checked-project-records) for what qualifies as a passing run.
[`upload_file` restore behavior](/docs/deployments/Databricks/Genesis-On-Databricks#durable-chat-uploads).
#### **`update_job` task replacement and persistence checks**
`update_job` updates an existing Databricks Job by `job_id`. The action always
reads the complete Job back after writing it, so each call makes one write request
and one or more `jobs/get` requests, one per response page. Job reads use the Jobs
API 2.2 pagination contract so task, cluster, parameter, and environment arrays
are not silently limited to the first 100 elements.
| Parameter | Type | Default | Description |
| - | - | - | - |
| `job_id` | integer | none | Required. Databricks Job to update. |
| `params` | object | none | Required. The new settings and update controls. |
| `params.mode` | string | `merge` when `params.tasks` is omitted | Use `merge` to merge submitted tasks into existing tasks by `task_key`, or `replace` to overwrite the whole Job definition through the Jobs API reset operation. If `params.tasks` is present, specify `mode` explicitly. |
| `params.tasks` | array of objects | omitted | Task definitions, each with a non-empty `task_key`. In `merge` mode, omitted tasks remain in the Job; in `replace` mode, the submitted task list becomes the Job's task collection. `tasks: []` is valid only with `replace`. |
| `params.fields_to_remove` | array of strings | omitted | Top-level setting names to unset. `null` is rejected. This is not usable with `mode='replace'`; use the replacement definition instead. Task collection and task removal entries are checked after the write. |
| `params.include_persisted_definition` | boolean | `false` | Include the exact Job definition already fetched for verification in `persisted_definition`, without making a second read request. |
The read-back compares only the fields that can be verified from the response:
persisted task keys, submitted task parameter containers, the top-level
`name`, `timeout_seconds`, `max_concurrent_runs`, and `parameters` settings, and
requested removals. The result places these under `verification.checked` and
returns `success: false` with named `divergences` when a checked value differs.
Other submitted fields—including notebook paths, task types, dependencies,
clusters, libraries, schedules, Git source, `run_as`, tags, notifications, and
environments—are not verified; call `get_job` when the complete definition
matters.
If the write succeeds but the read-back is incomplete or unusable, the result
uses `verification.status: 'unverifiable'` and `write_issued: true` rather than
claiming success. A response without a task collection is not treated as an
empty collection. Invalid task entries, duplicate `task_key` values, invalid
`fields_to_remove`, duplicate job parameters, and other rejected claims fail
before writing when they cannot be verified safely.
Examples:
```text theme={null}
databricks_action(
action="update_job",
job_id=123,
params={
"mode": "merge",
"tasks": [{"task_key": "daily_sales", "notebook_task": {"base_parameters": {"date": "today"}}}],
},
)
databricks_action(
action="update_job",
job_id=123,
params={
"mode": "replace",
"tasks": [],
"include_persisted_definition": True,
},
)
```
#### **`list_clusters` compute scope**
`list_clusters` lists the workspace's clusters through the Databricks clusters
API. It accepts the standard `connection_id` parameter; it does not accept a
parameter that enables or checks serverless compute.
* Each cluster includes `cluster_source`. When Databricks does not provide that
field, the tool returns `"UNKNOWN"`; do not treat it as an all-purpose cluster.
* `serverless_compute.status` is `"unknown"` because this API does not report
whether serverless is enabled. An empty or all-terminated cluster list does
not mean that notebooks cannot run.
* For serverless job compute, call `run_notebook` or `run_notebook_live` without
`cluster_id`. For `run_notebook`, passing the literal `cluster_id="serverless"`
selects the same serverless Jobs path; do so only when serverless execution has
been explicitly approved. Use `list_jobs` and `check_run_status` to see whether
runs have already succeeded through that path.
* SQL warehouses are a separate compute path listed by `get_workspace_info`.
They can serve SQL but cannot run a notebook. Check these paths before
reporting that no compute is available.
#### **`execute_python` process isolation and recovery**
`execute_python` runs in a separate Databricks execution process instead of the
Genesis server process. The process is created lazily for the first call and is
shared by sessions for the same `connection_id` and bot. This contains native
Spark Connect or gRPC crashes to that execution process, so the Genesis server
and other connections remain available.
On deployments where Genesis runs as root and the image provides the restricted
`cortex_sandbox` user (Snowflake, EKS and self-hosted Docker), the execution
process runs as `cortex_sandbox`, the same user `execute_python_code` runs as.
It gets no server credentials: its environment holds only `PATH`, locale, TLS
trust-store and proxy settings, and its own home directory. The connection's
token reaches it only for its Databricks session. Where Genesis cannot switch
users (Genesis Desktop, a local development server, or any server not running
as root), it runs as the server's own user, as before.
Because `execute_python` reuses its worker across calls, Python child processes
can remain active after a call returns; they end when Genesis tears down that
worker.
In a Databricks App, sessions are also isolated by signed-in user because the
connection runs with that user's forwarded token. A call without a signed-in
user cannot create or use a session. Each signed-in user can have at most 2
execution processes, with at most 8 across all App users. If either limit is
reached, session creation returns an error; wait for an idle session to expire
after 10 minutes before trying again. The App bundle must
request the `sql` user scope for Databricks connection SQL; see
[Databricks connection authentication](/docs/Integrations/Data%20Sources/Databricks/Databricks#choose-an-authentication-mode).
* Use `connection_id` to select the Databricks workspace. `execute_python` uses
serverless compute automatically; do not pass `cluster_id` for this action.
* Use `session_id` to preserve variables between calls. If it is omitted, the
response creates a session and returns its `session_id`. Sessions expire after
10 minutes without use; check `session_expires_at` or use
`list_python_sessions` to inspect the current thread's sessions.
* When a new session is created because `session_id` was omitted, was not found,
or had expired after more than 10 minutes of inactivity, the response includes
`session_note`. This is normal session recycling, not a context limit or an
error. If the previous session expired or the supplied ID was missing, rerun
the setup code; when no ID was supplied, save the returned `session_id` for
subsequent calls.
* A normal Python exception is returned in the result with `success: false`,
`error`, `error_type`, and traceback details. A crashed execution process is
returned with `error_type: "SparkWorkerCrashed"` and `lost_sessions`; every
session listed there has lost its variables.
* If the worker fails during startup, its captured output may include
`spark worker entrypoint reached pid=...`. This means the child reached its
entrypoint; if the marker is absent, the process may have failed before then,
so investigate worker startup or spawning.
* After `SparkWorkerCrashed`, omit `session_id` to create a fresh session and
rerun the setup code. The response explains that the Genesis server was
unaffected; do not treat this result as a server outage.
Example recovery call:
```python theme={null}
databricks_action(
action="execute_python",
code="result = spark.sql('SELECT COUNT(*) FROM catalog.schema.table').collect()",
connection_id="conn_id",
)
```
#### **Command execution and confirmation**
`execute_command` accepts `command` with an explicit `language`; an existing
context uses `execute_command_in_context` with `command` and `context_id` and
sends no language. SQL reads sent with `language="SQL"` do not need confirmation.
Python, Scala, and R commands require the user to approve code that may write
data before retrying with `params={"confirm": True}`. Direct SQL writes are
refused by these command actions even with confirmation; use
`run_sql_statement` for SQL writes instead.
Because `execute_command_in_context` sends no language, SQL text that is
classified as a read can run without confirmation, while ambiguous read-like
text requires confirmation.
For example, after the user approves running Python that may write data, submit
without waiting and use the returned `command_id` and `context_id` to check or
cancel it:
```python theme={null}
databricks_action(
action="execute_command",
cluster_id="cluster_id",
language="PYTHON",
command="spark.sql('INSERT INTO catalog.schema.table SELECT * FROM catalog.schema.staging')",
params={"confirm": True},
wait_for_result=False,
)
```
#### **Stopping remote Databricks work**
When a user stops a polling action, Genesis also stops the work already
running in Databricks instead of only ending local status polling:
* `run_notebook_live` cancels the Databricks job run identified by
`databricks_run_id`.
* `run_notebook_cells` and `execute_command` cancel the active command on its
cluster using `cluster_id`, `context_id`, and `command_id`.
* `run_sql_statement` cancels the SQL Statement API request it is polling. This
also applies when a spec-author seat's hard-stop stops the tool call. The
statement cancellation is best effort; the tool reports the stop even if
Databricks does not confirm cancellation.
* Genesis retries the remote cancellation once after a one-second delay. The
tool still reports the user stop even if both cancellation attempts fail.
* For `execute_command` with `wait_for_result`, Genesis destroys the temporary
execution context after the stop, including when cancellation fails.
If remote cancellation cannot be delivered, the result names the affected job
run, command, or statement and states that it may still be running. Use
`check_run_status` for a job run, `cancel_command` with the returned command
identifiers, or `check_sql_statement` with the statement id to inspect remaining
work.
This cancellation applies when the tool call is stopped, not when SQL polling
reaches its ceiling: a statement still running at `max_poll_seconds` returns
`timed_out: true` and remains running in Databricks. Use
`check_sql_statement` with its `statement_id` to inspect it.
#### **Uncertain notebook submission**
If `run_notebook` receives a transport failure after submission may have reached
Databricks, it returns `success: false` with `submission_state: "uncertain"`.
Treat that as an unknown outcome, not a failed run: inspect recent Jobs runs for
the notebook, use the matching run id with `check_run_status` if one exists, and
submit again only after confirming that no run was created. This avoids duplicate
billable work.
#### **SOURCE notebook structure warnings**
When `create_notebook` imports a notebook with `format="SOURCE"`, the tool
checks the finalized payload before sending it to Databricks. The import remains
non-blocking: the notebook is still sent, but the response includes
`structural_warnings` when the payload may have lost its structure.
If the supplied content is only one complete Markdown code fence, the tool
removes the fence and imports its body. An unlabelled fence is accepted, as are
fences labelled for the selected notebook language (`python`, `py`, `python3`,
`sql`, `scala`, or `r`). Content that already has a Databricks notebook header is
left unchanged.
The tool does not guess when the content is ambiguous. If the payload contains
multiple fenced blocks, text outside a single fence, or a fence labelled for a
different notebook language, it imports the content verbatim with the fence
markers and adds an actionable `structural_warnings` entry. Send only the
notebook code or one complete fence with the matching language label to avoid
this warning.
Warnings identify:
* A cell separator that is not exactly the language-specific marker on its own
line, such as `# COMMAND ----------` for Python or `-- COMMAND ----------`
for SQL.
* A Databricks magic directive such as `%md`, `%sql`, or `%python` that is not
prefixed with the language-specific `MAGIC` marker.
* An empty cell caused by a lost or duplicated cell separator.
Each warning includes the affected line or cell number and a description of the
issue. Inspect the generated notebook in Databricks when `structural_warnings`
is present.
For `SOURCE` imports that contain a `# MAGIC %md` (or language-specific equivalent)
cell, the tool also exports the imported notebook in `JUPYTER` format and compares
the stored `cell_type` values with the intended source cells. This catches code
that Databricks stored as markdown when a markdown block was not closed with a
cell separator. The import still succeeds, but the response adds a warning that
the affected code will not execute and that the notebook can report `SUCCESS`
without running it. If the JUPYTER export fails, is empty, or cannot be parsed,
the response instead says that cell-type verification could not be completed;
export the notebook as `JUPYTER` and inspect its `cell_type` values.
`sync_from_git` applies the same structure and cell-type checks to repository
notebooks. With the default `validate_transfer=True`, `success` is `false` when
any file is not validated, and `unvalidated_files` lists those repository paths.
Each entry in `validation_results` can include `cell_types` with one of these
values:
* `VERIFIED`: The stored cells match the source, or the notebook has no Markdown
cell that could cause this type of drift.
* `DRIFTED`: An intended code cell was stored as Markdown. The path is also
listed in `drifted_files`; this is the only result that establishes the code
will not execute, even if a job reports `SUCCESS`.
* `STRUCTURE_LOST`: Databricks stored a different number of cells than the
source sent. The path is listed in `structure_lost_files`, and which cells
execute is not established.
* `UNVERIFIED`: The JUPYTER export check could not run. The path is listed in
`unverified_files`, and executability is unknown rather than known-bad.
`FAILED` without `cell_types: DRIFTED` means only that the workspace copy does
not match the source; it does not establish whether the notebook executes.
`structural_warnings` is included when a notebook-structure finding is raised,
including warnings collected across the sync. The JUPYTER round trip costs one
additional export request for a notebook containing `# MAGIC %md` (or the
language-specific equivalent); notebooks without that marker skip the check.
The cell-type check runs even when `validate_transfer=False`, but transfer
validation status fields are only produced when transfer validation is enabled.
#### **`stage_from_s3` AWS credentials**
When `params.aws_credential_name` / `params.aws_connection` is omitted,
`stage_from_s3` resolves the caller's scoped default **AWS** connection (**My
default**, then **Role default**). If that default resolves but its secrets
cannot be loaded, Genesis returns an error instead of silently using ambient
host credentials. When no scoped AWS default applies, Genesis may fall back to
ambient `boto3` credentials and logs a warning — set a user or role default on
an AWS connection in **Config → Connections**, or pass an explicit
`params.aws_credential_name`, for predictable S3 access. See
[Connections — AWS connection modes](/docs/Setup/Connections#aws-connection-modes)
and [Default connection layers](/docs/Setup/Connections#default-connection-layers).
#### **SQL Safety for `run_sql_statement` and `run_sql_batch`**
The shared SQL safety and execution-outcome contract is in
[Connections](/docs/Setup/Connections#database-sql-safety); it also covers
`execute_command_in_context`.
| **Parameter** | **Applies to** | **Type** | **Default** | **Description** |
| - | - | - | - | - |
| **`params.statement`** | `run_sql_statement` | String | none | Required single SQL statement. `run_sql_statement` rejects semicolon-separated multi-statement batches. |
| **`params.sql_file_ref`** | `run_sql_statement` | Object | none | Instead of `params.statement`, reference a saved SQL file with required `repo_id` and `file_path`, plus optional `commit`. The file text runs unmodified through the same safety gate, confirmation, and Action Review. |
| **`params.expected_sha256`** | `run_sql_statement` | String | none | Optional SHA-256 of the saved file's bytes; the call is refused if it does not match. Applies only with `params.sql_file_ref`. |
| **`params.statements`** | `run_sql_batch` | Array\[String] | none | Required ordered SQL list. Maximum 50 statements per call. |
| **`params.confirm`** | Both SQL actions | Boolean | `false` | Re-call a previously blocked destructive statement such as `DROP TABLE`, `DELETE`, `TRUNCATE`, or `CREATE OR REPLACE TABLE` after review. |
| **`params.user_override`** | Both SQL actions | String | `""` | Re-call a blocked catastrophic namespace operation such as `DROP DATABASE`, `DROP SCHEMA`, `DROP CATALOG`, or `CREATE OR REPLACE DATABASE/SCHEMA/CATALOG` after explicit user approval. The value must exactly match the returned `statement_type`, such as `DROP SCHEMA`. |
Use `params.sql_file_ref` instead of `params.statement` or `params.sql` when the
statement is in a saved file. The shared reference format, optional hash check,
and receipt are documented under
[`_query_database`](/docs/Genbot-Toolkit/Data-Connector-Tools#_query_database).
The receipt always identifies the saved file; it includes `executed_sql_sha256`
and a text preview only when submission is proven by a returned `statement_id`.
Without those fields, execution is unproven. The file runs as one statement;
semicolon-separated batches remain unsupported.
If a safety refusal asks you to confirm or authorize a saved-file statement,
re-issue `run_sql_statement` with the same `params.sql_file_ref` and the requested
flag inside `params` (for example, `params.confirm: true` or the exact
`params.user_override` value). Do not retype the file's SQL. If the file contains
multiple statements, save each statement separately and run each with its own
`params.sql_file_ref`.
Use `run_sql_statement` for Databricks writes that `_query_database` refuses.
Privilege changes are governed writes too. `GRANT`, `REVOKE`, `SET TAG`, and
`UNSET TAG` are classified as privilege changes and return the normal
confirmation response; review the returned `classification` and `sql` before
retrying. Command Execution requests that contain SQL are classified from the
command text rather than only from the caller's language label, so a command
that hides a write is not treated as Python-only execution.
It accepts one statement per call; review the returned classification before
retrying with `params.confirm: true` for a table-level destructive statement.
`params.confirm` does not authorize catastrophic namespace operations, which
require the exact `params.user_override` value described above.
`execute_python` applies the same table-discard safeguard to code containing
`DROP TABLE`, `CREATE OR REPLACE TABLE`, or `REPLACE TABLE`. Those operations are
refused unless the user has agreed to discard the current table data and the
call is repeated with `params={"confirm": true}`. For row removal, use
`DELETE FROM ... WHERE ...` or a `MERGE` instead.
If Databricks SQL returns `requires_confirmation: true`, review the returned
`classification` and `sql` before retrying. `params.confirm` does not
authorize `CATASTROPHIC` operations. For `run_sql_batch`, Genesis
reclassifies each statement independently, so one `params.user_override`
authorizes only the matching catastrophic operation type. Split namespace
changes into separate statements when you need explicit approval.
When the optional SQL intent-match guard is enabled, `run_sql_statement` can
also return `requires_confirmation: true` with `severity: intent_mismatch`
if the planned statement does not clearly match the user's latest request.
`run_sql_batch` still evaluates each statement separately before execution.
The correct recovery is a fresh chat reply that explicitly names the exact
operation and target; if the statement is still destructive or catastrophic,
keep `params.confirm` or `params.user_override` on the retry as required.
`CREATE OR REPLACE VIEW ... WITH METRICS` is classified as destructive even
when the target name is new, so review the confirmation instead of assuming
that a new name makes the statement non-destructive. `DESCRIBE TABLE EXTENDED ... AS JSON`
is a read, but the local SQL parser can classify its `AS JSON` suffix as unknown
and request confirmation; treat that prompt as parser caution before retrying.
#### **Unity Catalog metric views**
Use `list_metric_views` to identify metric views in one Unity Catalog schema. The
documented `TABLE_TYPE` values for `information_schema.tables` do not include
`METRIC_VIEW`, although some workspaces return that value there anyway. `SHOW
VIEWS` and the Databricks JDBC driver do not provide a consistent discriminator,
so this action uses the Unity Catalog REST catalog's `table_type` value instead.
It requires `params.catalog` and `params.schema`, and uses the selected
`connection_id` (or the configured Databricks environment credentials when
`connection_id` is omitted).
```text theme={null}
databricks_action(
action="list_metric_views",
connection_id="my_databricks",
params={"catalog": "main", "schema": "analytics"}
)
```
The result includes each visible metric view's `full_name`, `name`, `table_type`,
`owner`, `comment`, `updated_at`, and `properties`, plus `returned_count`,
`objects_examined`, and `pages_read`. Read `complete` before treating the result
as exhaustive: `true` means the listing reached the end of the schema, while
`false` means the result is only a prefix after the action's page limit. A
successful empty `metric_views` list means no metric views are visible to the
calling identity in that schema; a failed request returns `success: false` rather
than an empty list. The action performs one Unity Catalog HTTPS request per page
and does not use a SQL warehouse.
When querying a metric view, wrap every measure in `MEASURE()`, for example
`SELECT MEASURE(\`Total Revenue\`) FROM main.analytics.revenue\_metrics`.
A bare measure reference can return a plausible number without raising an
error, but it is not the measure as defined by the semantic layer. To recover
the metric-view definition, use `DESCRIBE TABLE EXTENDED catalog.schema.metric\_view AS JSON`;
its `view\_text`contains the YAML definition.`information\_schema.views\` does
not reliably return metric views.
#### **Unity Catalog registration for the semantic layer**
Genie Ontology is derived from Unity Catalog metadata rather than directly
authored by this tool. These actions register the descriptions, tags,
certification, informational keys and domains that Unity Catalog exposes to
Catalog Explorer and Genie. They use the selected Databricks connection and
SQL warehouse for SQL operations; domain and tag-policy actions use the REST
API instead.
Use `register_uc_assets` to register several tables in one schema. It is the
batch form of the comment, tag and key actions and also records migration
provenance on the real Unity Catalog table, not as a separate shadow object.
Pass `params.dry_run=true` first for a large registration: every write
statement is returned without being sent. A live registration runs one warehouse
statement per comment, tagged object and constraint, and key verification may
also scan tables. When a table-comment decision depends on the comment already
on the table, both a dry run and a live run send one `information_schema.tables`
read per call to fetch the existing comments.
`COMMENT ON TABLE` replaces the whole comment, so existing table comments are
preserved by default by these Unity Catalog actions. A raw `COMMENT ON` sent
through `run_sql_statement` replaces the whole comment; use
[`apply_uc_comments`](/docs/Genbot-Toolkit/Databricks-Action-Tools#unity-catalog-registration-for-the-semantic-layer)
to append a note after the live comment. Provenance alone writes only the
`genesis_*` provenance tags, not a comment. Use `params.comment_mode` to change that:
| `comment_mode` | Provenance without `comment` | Explicit `comment` |
| - | - | - |
| `preserve` (default) | No comment is written; provenance goes to tags. | Never removes existing text: appended after the existing comment (separated by a blank line, reported as `table_comment_appended`), written as given when it carries the existing text on at a word or paragraph boundary, and skipped when its paragraphs already appear in the existing comment as a contiguous, ordered run, so a re-run adds nothing. A provenance sentence already on the table stays last, and one recording a different source is kept. A write that would grow the comment past 2048 UTF-8 bytes (an appended description, a first description, or an added provenance sentence) is held back. If the existing comment could not be read (the read failed or returned no row for the table), it is held back and listed in `comments_held_back`. |
| `append` | The provenance sentence is added after the existing comment, once; a re-run does not add it again. | Same as `preserve`. |
| `replace` | The provenance sentence replaces the existing comment. | Written regardless of the existing comment. |
A comment consisting only of an earlier provenance sentence for the same source is
treated as generated provenance, not authored guidance: it is recomposed under
`append`, and an explicit comment replaces it rather than being appended to it.
Column comments (`assets[].columns`) and `params.schema_comment` follow the same
rules as an explicit table comment, without a provenance sentence: under `preserve`
and `append` a note is appended after the existing comment, skipped when it is already
there, and held back when the existing comment could not be read; only `replace`
overwrites. Deciding them costs one `information_schema.columns` read when any asset
carries column comments and one `information_schema.schemata` read when
`schema_comment` is given, in a dry run too. `schema_comment_written` indicates whether
a schema comment was included for writing, and `schema_comment_appended` says whether
it was appended; `existing_schema_comment_error` reports a failed schema-comment read.
For each asset, `column_comments_written` counts comment statements sent (or composed
in a dry run), while `column_comments_appended`, `column_comments_held_back`, and
`column_comments_replaced` report the per-column outcomes. The top-level
`existing_column_comments_error` reports a failed column-comment read.
Because no mode but `replace` removes text, a correction or rewording of an existing
comment needs `comment_mode: "replace"` with the full corrected text. Sent under the
default, it is appended beside the old wording. Matching is by whole paragraph: a short
note that only appears inside a longer sentence is still appended.
Every comment write that would lose existing comment text is listed with
`existing_length` and `new_length` (a table entry names its `table`, a column entry its
`table` and `column`, a schema entry its `schema`): in `would_replace_existing_comment` for a
dry run and in `replaced_existing_comment` for a live run. A null
`existing_length` means the existing comment could not be read. A write that fails is reported in `failures`, not in these lists. An unrecognised field on an asset, such as `assets[].comment_mode`, is refused naming it. Check this list
before applying a registration; an empty `failures` list does not mean no
text is lost.
| Parameter | Type | Default | Description |
| - | - | - | - |
| `params.catalog`, `params.schema` | string | required | Unity Catalog schema to register. |
| `params.assets` | array of objects | required | Each object has `table` and optional `comment`, `columns` (column-to-description map), `tags`, `column_tags` (column-to-tag-map), `certify`, `provenance`, `primary_key`, and `foreign_keys`. |
| `params.assets[].provenance` | object | omitted | Optional `source_system`, `source_path`, `source_id`, `migrated_at`, `proof`, and `migration_run`. Always written as queryable provenance tags; rendered in the table comment only beside an explicit `comment` or under `comment_mode` `append` or `replace`. |
| `params.comment_mode` | string | `preserve` | How to treat an existing table comment: `preserve`, `append`, or `replace`. See the `comment_mode` table above. |
| `params.assets[].primary_key` | object | omitted | `{name, columns[], rely?}` candidate primary key. `rely` defaults to `params.rely`; a value here wins over it. |
| `params.assets[].foreign_keys[]` | object | omitted | `{name, columns[], references_table, references_schema?, references_columns[]?, rely?}`. `references_schema` defaults to the registered schema; when provided, `references_columns` must match the child-column count. `rely` defaults to `params.rely`; a value here wins over it. A key spec carrying any other field is refused naming it, rather than accepted and ignored. |
| `params.common_tags`, `params.schema_tags` | objects | omitted | Tags shared by every table, or applied to the schema. |
| `params.schema_comment` | string | omitted | Comment to write on the schema, appended after an existing schema comment unless `comment_mode` is `replace`. |
| `params.dry_run` | boolean | `false` | Return planned write statements without executing them. When existing comments must be checked, it resolves a warehouse and sends those reads (up to three: tables, columns, schemata). |
| `params.warehouse_id` | string | connection's configured warehouse | Select the SQL warehouse for this action. |
| `params.verify` | boolean | `true` | Verify candidate keys against table data before declaring them. Set `false` to declare based on caller-provided evidence instead. |
| `params.rely` | boolean | `false` | Add `RELY` to declared constraints. Boolean text values `true`, `t`, `yes`, `y`, `on`, `1`, `false`, `f`, `no`, `n`, `off`, and `0` are also accepted, case-insensitively; other values are rejected. Use only when the constraints are true; the optimizer may rely on them. |
| `params.allow_schema_change` | boolean | `false` | Permit the `SET NOT NULL` schema change required for a primary key. Without it, a primary key is reported as skipped. |
Primary and foreign keys are informational, not enforced by Unity Catalog.
Primary-key columns must become `NOT NULL`, so the action requires
`allow_schema_change=true` to make that schema change. Foreign keys are declared
after all primary keys so an asset's order does not determine whether its
references can be registered. Inspect `totals.keys_not_declared` and each
key's `skipped_reason`; `success` reports SQL execution, not whether every
candidate key was accepted.
```text theme={null}
databricks_action(
action="register_uc_assets",
connection_id="my_databricks",
params={
"catalog": "main",
"schema": "analytics",
"dry_run": true,
"assets": [
{
"table": "orders",
"comment": "Customer orders",
"columns": {"order_id": "Unique order identifier"},
"provenance": {"source_system": "snowflake", "source_path": "SALES.ORDERS"}
}
]
}
)
```
For one table, `apply_uc_comments` takes `params.catalog`, `params.schema`,
`params.table`, and optional `params.comment`, `params.columns` (a map of
column names to comments), `params.provenance`, and `params.comment_mode`. It
executes one warehouse statement per comment, plus one `information_schema.tables`
read when deciding a table comment; that read also runs in a dry run. With
`params.dry_run` (default `false`), it returns the write statements without sending
them. Use it for governance or annotation notes: `comment_mode` defaults to
`append` here, so `params.comment` is added after the existing table comment once,
and re-running the same note writes nothing. `replace` overwrites. The result reports
`table_comment_appended` and `comment_held_back`; `existing_comment_error` identifies
a failed comment read. If a write replaces existing text, the result names it in
`replaced_existing_comment`, or `would_replace_existing_comment` in a dry run.
An empty column comment is refused, naming the column. Column comments get the same
treatment per column, and `column_comments_written` counts comment statements sent
(or composed in a dry run), at the cost of one
`information_schema.columns` read: the result lists `column_comments_appended`,
`column_comments_held_back` and `replaced_existing_column_comments` (or
`would_replace_existing_column_comments`), and `existing_column_comments_error` names a
failed read. `params.warehouse_id` optionally selects the
warehouse; otherwise the connection's configured warehouse is used.
`apply_uc_tags` accepts `params.catalog` and `params.schema`, optional
`params.table`, `params.schema_tags`, `params.tags`, `params.column_tags`,
`params.certify` (boolean, default `false`), `params.dry_run` (boolean, default
`false`), and `params.warehouse_id` (otherwise the connection's configured
warehouse). It runs one warehouse statement per tagged schema, table or column.
`certify=true` adds the governed tag `system.certification_status=certified`
to the table.
Dry runs that compose tag writes—including `register_uc_assets`, `apply_uc_tags`,
and `assign_to_domain`—do not check account tag policies. Their response includes
`tag_policies_checked: false` and a `tag_policies_note`; a governed key or a
value disallowed by its policy can still be refused when the statements run.
Use `list_tag_policies` to inspect the policies. `declare_uc_keys` takes
`params.catalog`, `params.schema`, `params.table`, optional `params.primary_key`
and `params.foreign_keys[]` in the shapes above, and the `verify`, `rely`,
`allow_schema_change`, `dry_run` (default `false`), and `warehouse_id` controls.
`warehouse_id` defaults to the connection's configured warehouse. It scans the
table once per candidate key and runs one statement per constraint, so it can
incur warehouse compute. Verification is on by default, and a key whose data
cannot be verified is not declared.
`params.rely` is only the default for the call. `params.primary_key` and each
`params.foreign_keys[]` entry may state its own `rely`, and the narrower
statement wins, so one call can trust four measured keys and withhold `RELY`
from a fifth. `RELY` licenses the query optimiser to rewrite queries assuming
the constraint holds, so each key's entry in the result reports the `rely`
actually emitted on it -- read that rather than `success`, which only covers the
statements that were sent. The `rely` value accepts the boolean text spellings
listed in the `register_uc_assets` parameter table above; other values are
rejected. A key spec carrying a field the action cannot act on is refused
naming it rather than accepted and ignored.
Use `describe_uc_registration` before or after publishing. It requires
`params.catalog` and `params.schema`, and optionally filters to `params.table`.
It makes four aggregate `information_schema` reads on the selected warehouse
(the connection's configured warehouse unless `params.warehouse_id` is given):
tables, columns, table tags and constraints. Check
`summary.counts_complete` before using the column totals; unreadable counts are
named in `summary.unreadable_count_rows`. The call costs four warehouse reads;
`params.dry_run` is not supported for this read action.
`create_metric_view` creates or replaces a metric view with one warehouse
statement. It requires `params.catalog`, `params.schema`, `params.name`, and
`params.spec`; optionally pass `params.comment`, `params.dry_run` (default
`false`), and `params.warehouse_id` (otherwise the connection's configured
warehouse is used). `spec` requires `source` and at least one measure. Its
optional `dimensions[]` and `measures[]` entries each have `name` and `expr`,
with optional `window` and `partition`. Optional `joins[]` entries have
`name`, `source`, and either `on` or `using[]`; declared foreign keys are not
inferred. The action generates YAML from this structured spec rather than
accepting YAML or SQL text. View-level descriptions use `params.comment`;
dimensions and measures cannot carry comments. The result includes a
`query_hint` using `MEASURE()` for the first measure. `list_metric_views`
identifies metric views; see the section above for its listing behavior.
#### **Databricks Genie spaces and semantic-layer binding**
These `databricks_action` actions manage Genie spaces directly; they are separate
from `mcp_admin_tools`' runtime `add_genie_space`, which exposes an existing Genie
space as MCP tools. Register Unity Catalog comments before binding: Genie space
table entries have no column list, so column meaning comes from Unity Catalog
comments. See the registration actions above.
`bind_semantic_layer_to_genie` discovers tables and metric views in a schema,
copies each table comment to its Genie description, and creates a new space or
updates an existing one. It requires a SQL warehouse for one `information_schema`
query; creating a space also uses that warehouse as the space's query warehouse.
An existing space keeps its own warehouse unless `params.warehouse_id` is passed.
The default binds all discovered tables; use `params.only_metric_views=true` to
bind only metric views. `params.verify_question` optionally asks the bound space
and returns its generated SQL and answer, incurring Genie planning and warehouse
compute in addition to discovery and the write. A verification failure does not
undo a successful binding.
| Parameter | Type | Default | Description |
| - | - | - | - |
| `params.catalog`, `params.schema` | string | required | Unity Catalog catalog and schema to discover. Each must be a plain identifier containing letters, digits, and underscores, and starting with a letter or underscore. |
| `params.space_id` | string | omitted | Existing Genie space to update. If omitted, creates a space. |
| `params.title` | string | `. semantic layer` | Title for a new space; optionally renames an existing space. |
| `params.only_metric_views` | boolean | `false` | Bind only metric views rather than all tables and metric views discovered in the schema. |
| `params.mode` | string | `merge` | For an existing space, `merge` preserves sections other than data sources that this action writes; `replace` removes unnamed sections. This action always replaces the entire data-source list with the discovered schema's list. |
| `params.instructions` | string or string\[] | omitted | Optional instruction text or lines to include with the binding. |
| `params.sql_expressions` | object\[] | omitted | Optional reusable SQL snippets; entries use `display_name`, required `sql` (string or string\[]), and optional `comment` and lowercase 32-character hex `id`. |
| `params.benchmark_questions` | object\[] | omitted | Optional evaluation questions; entries require `question` and `answer_sql` (or `sql`), and may include a lowercase 32-character hex `id`. |
| `params.warehouse_id` | string | connection's resolved warehouse | Warehouse for schema discovery and, when creating, the new Genie space. Existing spaces retain their warehouse unless this parameter is supplied. |
| `params.verify_question` | string | omitted | Ask the space after binding; the result's `verified` is true only when the question returns successfully. |
| `params.timeout_seconds` | integer | `180` | Verification wait limit, from `1` to `900` seconds. |
| `params.parent_path` | string | omitted | Parent workspace path for a newly created space. |
| `params.dry_run` | boolean | `false` | Discover the schema and return the planned payload without creating or updating a space. |
Check `discovery.tables_without_comments` and `discovery.descriptions_complete`
to see whether all bound tables have comments. `discovery.tables_skipped_unnameable`
and `discovery.schema_fully_bound` report table names that could not be represented
safely. A schema with more than 200 discovered data sources is refused; use the
returned names with `create_genie_space` or `update_genie_space` to select a
smaller set.
```text theme={null}
databricks_action(
action="bind_semantic_layer_to_genie",
connection_id="my_databricks",
params={
"catalog": "main",
"schema": "analytics",
"only_metric_views": true,
"verify_question": "Which carrier has the worst on-time performance?"
}
)
```
Use `create_genie_space` to author a space from explicit sources instead of
discovering a whole schema. It requires `params.title` and at least one of
`params.tables`, `params.metric_views`, or `params.volumes`; table and metric-view
entries are strings or `{identifier, description}` objects. Optional content
includes `params.instructions` (text or lines), `params.sql_expressions[]`
(`display_name`, `sql`, optional `comment`), and `params.benchmark_questions[]`
(`question`, `answer_sql`). It resolves `params.warehouse_id` from the explicit
value or the selected connection and makes one HTTPS POST; no compute is billed
until the space is asked a question. Set `params.dry_run=true` to inspect the
payload without writing.
`list_genie_spaces` accepts `params.page_size` (default `100`, maximum `1000`)
and `params.title_contains` (case-insensitive, client-side filter). It makes one
HTTPS GET per page, up to 50 pages; check `complete` and `unreadable_rows` before
treating the returned list as exhaustive. `get_genie_space` requires
`params.space_id`, makes one GET, and returns the parsed `serialized_space` and
`etag` for a later guarded update.
`update_genie_space` requires `params.space_id` and an explicit
`params.mode` (`merge` or `replace`). It may also update `params.title` or
`params.warehouse_id`, or content sections: `params.tables`, `params.metric_views`,
`params.volumes`, `params.instructions`, `params.sql_expressions`, and
`params.benchmark_questions`. In `replace` mode, content this call omits is
removed. In `merge` mode, the action reads the current space and rewrites only
named sections, preserving the others; it sends the read etag so concurrent
changes are refused. A merge normally costs one GET plus one PATCH. If the
workspace refuses that etag with `modified since this export was taken`, the
action re-reads and re-merges once, then retries with the fresh etag. If that
write is refused too, it reads once more: an unchanged etag allows one final
PATCH without an etag, while a changed etag stops the update without writing.
The result's `etag_conflict` explains a retry; an edit saved between the final
read and an etag-less write would not be kept. Retrying can add up to two GETs
and two PATCHes. A caller-supplied `params.expect_etag` is never retried past.
Replace costs one PATCH. For an explicitly guarded replace, pass
`params.expect_etag` from `get_genie_space`. Both write modes accept
`params.dry_run=true`.
`delete_genie_space` requires `params.space_id` and `params.confirm_title`, which
must exactly match the current title. It reads the live title before deleting;
`params.dry_run=true` performs that check without deleting. The action cannot
establish who created a space, so only delete spaces you own.
`ask_genie` requires `params.space_id` and `params.question`. It runs a Genie
planning pass and the generated SQL on the space's warehouse, typically taking
20–60 seconds and billing compute. The result includes `generated_sql`,
`answer`, and a sample of result rows; `params.include_rows` defaults to `true`
and `params.max_rows` defaults to `50` (maximum `1000`).
`params.timeout_seconds` defaults to `180` (maximum `900`). A timeout returns
`success: false` with conversation and message IDs; Genie may still finish after
the wait ends. Stopping the tool cancels SQL execution if it has started, but
does not cancel Genie planning.
#### **Unity Catalog domains and governed tag policies**
Domains are a Public Preview capability behind the account-admin
“Domains and Discover Page” preview. The domain REST API is Beta. A domain
requires an account-level tag policy, and assigning an asset to a domain sets
that policy's governed tag on the asset. These operations use REST calls except
for `assign_to_domain`, which runs one warehouse statement per asset.
* `list_tag_policies` takes no action-specific parameters and returns the
account-level policies visible to the identity with one HTTPS request.
* `create_tag_policy` requires `params.tag_key`; optional `params.description`
and `params.allowed_values[]` define the policy; `params.dry_run` defaults to
`false`. Creating a policy writes an account-level object, not a
workspace-scoped setting. It makes one HTTPS POST and, when allowed values
are supplied, one PATCH to store them; inspect `allowed_values_stored` before
assigning values.
* `create_domain` requires `params.tag_key` for an existing policy. Optional
`params.subtitle`, `params.description`, `params.parent_domain_id` and
`params.icon` are sent with the domain; `params.dry_run` defaults to `false`.
It lists domains and then makes one HTTPS POST only when no domain exists on
that key. A listing error prevents creation.
* `list_domains` takes optional `params.parent_domain_id` to filter visible
domains and makes one HTTPS GET. A non-empty result carries
`visibility_limited: false` and reports the visible count. An empty result
carries `visibility_limited: true` and says that this identity cannot see any
domains, while some may exist; when filtering by parent, the message names that
parent. It does not prove none exist.
* `assign_to_domain` requires `params.tag_key`, `params.value` (one allowed by
its policy), and `params.assets[]`. Each asset has `catalog`, `schema`, and
optional `table`; omitting `table` assigns the schema. `params.dry_run`
defaults to `false` and returns statements without running them.
`params.warehouse_id` optionally selects the SQL warehouse; otherwise the
connection's configured warehouse is used.
#### **Unity Catalog Pages**
Unity Catalog Pages hold governed definitions of business concepts and are
read by Genie when published. Pages have no public Databricks API; these
actions use the Databricks console's undocumented, unsupported GraphQL API, which
can change without notice. Use them only for an attended, one-time glossary
seed or migration, not in scheduled or recurring work. Enable
`tools.databricks.unity_catalog_pages_enabled` (default `false`) in Genesis
configuration before calling them. The setting is read for each call, so turning
it off blocks the next call without a restart. When disabled, each action
refuses and sends nothing to the workspace. For configuration details, see
[Unity Catalog Pages authoring](/docs/Setup/Advanced-Settings#unity-catalog-pages-authoring).
A Page must belong to an existing Domain. Use `list_domains` to find its ID, or
create an account tag policy and Domain with `create_tag_policy` and
`create_domain` first. New Pages are `DRAFT`, invisible to users and unread by
Genie until published. All write actions accept `params.dry_run` (default
`false`) to inspect payloads without sending them.
| Action | Parameters | Behavior and cost |
| - | - | - |
| `bulk_create_uc_pages` | `params.pages` (required array of objects with `name`; optional `description`, Markdown `body`, `synonyms[]`, and per-page `domain_id`); `params.domain_id` (default domain); `params.chunk_size` (integer, default `25`, range `1`–`100`); `params.skip_existing` (boolean, default `true`); `params.dry_run` (boolean, default `false`) | Recommended for glossary imports. Creates Pages in batches, one HTTPS request per chunk, plus one workspace listing when skipping existing names. Duplicate matching uses domain and name; `created_page_ids` identifies created Pages, including on partial failure. Imports remain drafts; publish selected Pages separately. |
| `create_uc_page` | `params.name` and `params.domain_id` (required); optional `params.description`, `params.body`, `params.synonyms[]`, `params.publish` (boolean, default `false`), `params.skip_existing` (boolean, default `true`), and `params.dry_run` (boolean, default `false`) | Creates one Page with one HTTPS request. With the default `skip_existing`, a matching Page in the same Domain is returned rather than duplicated; checking requires a listing request. `publish=true` makes a second request to publish it. |
| `publish_uc_page` | `params.page_id` (required); `params.lifecycle_state` (`DRAFT` or `PUBLISHED`, default `PUBLISHED`); `params.dry_run` (boolean, default `false`) | Changes only the Page lifecycle state with one HTTPS request. `PUBLISHED` makes the Page visible to users and available to Genie. |
| `list_uc_pages` | Optional `params.domain_id` and `params.lifecycle_state` (`DRAFT` or `PUBLISHED`) | Reads the workspace listing (100 Pages per request, up to 50 pages). Both filters are applied client-side and do not reduce the listing cost. Check `complete`; `false` means the returned rows are only a prefix. |
| `delete_uc_page` | `params.page_id` and `params.confirm_name` (required; must exactly match the current Page name); `params.dry_run` (boolean, default `false`) | Reads the listing to confirm the live name, then deletes with a second HTTPS request. The API does not identify who authored a Page; delete only Pages you created. |
```text theme={null}
databricks_action(
action="bulk_create_uc_pages",
connection_id="my_databricks",
params={
"domain_id": "domain-id-from-list_domains",
"dry_run": true,
"pages": [
{"name": "Net Revenue", "description": "Revenue after refunds", "synonyms": ["net sales"]}
]
}
)
```
#### **Unity Catalog external metadata and lineage**
Use external metadata when the source system will remain alongside Databricks:
it creates foreign-system objects as Unity Catalog securables and can connect
them to Unity Catalog tables. For a migration from a system being retired, use
`register_uc_assets` to put provenance on the real Databricks table instead.
External metadata and lineage consume per-metastore limits of 10,000 objects
and 100,000 relationships. External lineage is visible in Catalog Explorer and
through this API, but is not written to `system.access.table_lineage` or
`system.access.column_lineage`.
`publish_external_metadata` requires `params.name`; optional fields are
`params.system_type`, `params.entity_type` (default `asset`), `params.description`,
`params.url`, `params.columns[]`, `params.properties`, and `params.dry_run`
(default `false`). Names are lowercased and may contain only letters, digits,
`-`, `_`, `:`, and `@`; periods, spaces and forward slashes are not accepted.
Supported `system_type` values are `AMAZON_REDSHIFT`, `AZURE_SYNAPSE`,
`CONFLUENT`, `DATABRICKS`, `GOOGLE_BIGQUERY`, `KAFKA`, `LOOKER`,
`MICROSOFT_FABRIC`, `MICROSOFT_SQL_SERVER`, `MONGODB`, `MYSQL`, `ORACLE`,
`OTHER`, `POSTGRESQL`, `POWER_BI`, `SALESFORCE`, `SAP`, `SERVICENOW`,
`SNOWFLAKE`, `STREAM_NATIVE`, `TABLEAU`, `TERADATA`, and `WORKDAY`. If omitted,
the type is `OTHER`. A create conflict is updated with a patch.
`list_external_metadata` optionally takes `params.name_prefix` and
`params.entity_type`; both filters are client-side, so all pages are still read.
`params.page_size` defaults to `1000` and is capped at `1000`. The result
includes `objects_examined`, `pages_read`, `complete`, and the observed
metastore budget. The listing stops after 50 pages; `complete=false` means it
is only a prefix. It makes one HTTPS GET per page, up to 50.
`delete_external_metadata` requires `params.name` and makes one HTTPS DELETE.
Delete lineage relationships first: removing an object can make its edges
unlistable without proving they were released from the relationship limit.
Lineage endpoints are objects such as
`{type: "external_metadata", name: "source:table:orders"}` or
`{type: "table", name: "main.analytics.orders"}`. The other supported types
are `path` (provide `url`, or use `name` as the URL) and `model_version`
(provide `name` and `version`). `publish_external_lineage` requires `params.source`
and `params.target`; optional `params.columns[]` contains `{source, target}`
column pairs, `params.properties` adds string properties, and `params.dry_run`
defaults to `false`. Declare external-object columns before publishing edges
that map those columns. Publishing makes one HTTPS POST; a duplicate edge is
reported unchanged rather than patched.
`list_external_lineage` requires `params.object_type` and `params.direction`
(`UPSTREAM` or `DOWNSTREAM`). For a path, pass `params.object_type="path"` and
`params.url`; other endpoint types use `params.object_name`, and a
`model_version` also needs `params.version`. Optionally pass `params.page_size`
(default `1000`, maximum
`1000`). The request reads one direction and returns `complete` to show whether
another page exists.
`delete_external_lineage` requires the original `params.source` and
`params.target` endpoint pair; it accepts `params.dry_run` (default `false`) and
makes one HTTPS DELETE when not in dry-run mode.
`publish_context_subgraph` publishes a set of Genesis graph nodes and data-flow
edges together. It requires `params.name_prefix` and `params.nodes[]`; pass
`params.edges[]` for graph edges, `params.default_system_type` as an optional
fallback, and `params.dry_run` (default `false`) to inspect all payloads before
sending requests. Each node uses its `qualified_name` and may include
`entity_type`, `description`, `url`, `columns[]`, `properties`, or `system_type`.
The prefix namespaces encoded names, and the full qualified name is retained in
the external object's properties. Optional `params.uc_table_targets` maps
qualified names to three-part Unity Catalog table names to add migration edges;
`params.column_maps` maps each qualified name to `[{source, target}]` pairs.
Each graph edge identifies nodes with `actor_id` (or `source`) and `target_id`
(or `target`), and may set `properties.edge_type` and `operation`.
Nodes are sent before edges. The result counts skipped placeholder nodes,
non-data-flow edges, and dangling edges. Only `data_flow` and `executes` graph
edge types are published (an edge without `properties.edge_type` defaults to
`data_flow`); `READS` edges reverse source and target to represent the direction
of data flow. Review `metastore_budget_consumed` before a large publish. Each
object and edge makes an HTTPS request (an existing object may require a second
PATCH); no SQL warehouse compute is used.
```text theme={null}
databricks_action(
action="publish_context_subgraph",
connection_id="my_databricks",
params={
"name_prefix": "migration-2026",
"dry_run": true,
"nodes": [
{"qualified_name": "snowflake://SALES.ORDERS", "columns": ["ORDER_ID"]}
],
"edges": []
}
)
```
#### **Lakeflow Designer visual flows**
A Lakeflow Designer flow is the canvas rendering of a pipeline: operators wired
together that a person opens and edits in the Databricks editor. On disk it is a
notebook whose path ends `.designer.ipynb`, which is what makes the workspace
store it as a `DESIGNER_FILE`; one notebook cell is one canvas node, and the
graph lives in a YAML docstring at the top of each cell. Genesis emits that graph
and the generated code body each cell carries beneath it. A generated flow has
been measured to open as a complete, editable canvas, including its operators,
names, edges, ports and positions. Whether a generated flow runs without being
edited first has not been established. Every result carries `run_readiness` with
that limitation; treat the flow as a drawing to review and edit, not as a
verified runnable pipeline.
`describe_designer_templates` returns the 16 canvas operators with their
`template_version`, input ports, output ports, ports that accept several inputs,
and default config; pass `params.template` for one of them. It reads a catalog
embedded in Genesis and makes no request. Call it before writing a flow
specification: the vocabulary is not guessable, and a wrong value produces a file
that opens as an empty canvas with no error. The palette's "Select" operator
serialises as template `transform`; a filter has two output ports,
`filtered_data` and `excluded_data`; a join's inputs are `left` and `right` while
the other operators use their listed ports; and a row limit is a string. Five palette
operators -- Notifications, Guardrails, Note, Group and Visualization -- have no
captured schema and are named as such rather than refused as unknown.
`generate_designer_flow` requires `params.name` and `params.nodes[]`, each
`{id, template, name?, config?, position?}`, and takes `params.edges[]` of
`{from, to, output_port?, input_port?}`. A port defaults to the template's first,
which is the kept rows for a filter. Edges may instead be embedded on a node as
`input[]`, the spelling `get_designer_flow` returns. Nodes without a `position`
are laid out left to right by dependency depth on the editor's own grid;
`params.layout` sets `column_spacing`, `row_spacing`, `origin_x` and `origin_y`,
and a node carrying `position` keeps it. The whole specification is validated
before anything is emitted -- unknown template, unreal port name, edge to a
missing node, a cycle, or a second edge into a port that takes one input -- so a
refusal means nothing was produced. The notebook is returned only with
`params.include_content=true`; `content_sha256` and `content_bytes` travel
either way. Nothing is sent to the workspace.
`render_pipeline_as_designer_flow` takes `params.pipeline` of `{name, steps[]}`,
each step `{id, kind, name?, inputs?}` plus that kind's own arguments: `table` or
`path` for a read, `condition` for a filter, `columns` for a select or a drop,
`renames` for a rename, `join_keys` for a join, `query` for SQL, `code` for the
`python` and `custom` kinds, and a three-part `table` for a write. Edges come from
`inputs`, and a join's two inputs land on `left` and `right` in the order given,
which is also the order its conditions address them. Supported step arguments are
composed into operator config. Other kinds render with the operator's default
config and are counted in `steps_with_default_config` and named in `warnings`;
pass `step.config` to set or override config explicitly. Missing required read or
write arguments are refused unless `step.config` supplies them. Nothing is sent
to the workspace.
`publish_designer_flow` takes everything either generator takes, or
`params.pipeline`, plus `params.path`. The designer extension is added when the
path lacks it, and the result reports the path written. The path is read first:
an existing flow that already matches is reported `unchanged` and not rewritten,
and an object that is not a designer file is refused rather than overwritten.
`result.url` opens the canvas at `/editor/designer-files/`, which is
not the workspace path URL. Publishing creates the containing folder if needed,
and makes 3-5 HTTPS requests plus one connection-details lookup for the canvas
link; re-publishing an existing flow adds an export to compare it. It starts no
compute.
`list_designer_files` requires `params.path` and takes `params.recursive`
(default `true`). The workspace listing has no type filter, so every object is
read and `objects_examined` reports how many; `complete=false` means the walk
reached its 200-directory ceiling and the rows are only a prefix. It makes one
HTTPS GET per directory, up to 200, and one connection-details lookup when flows
are found. `get_designer_flow` requires `params.path` and parses
each cell's YAML back into nodes and edges, dropping the `codegenHash` and
`previewCodeHash` values the editor stamps on a node it has evaluated; a cell
that does not parse is listed in `unparsed_cells` with its index and the reason,
and `complete` goes false. `delete_designer_flow` requires `params.path` and
reads the object type first, refusing anything that is not a designer file. Get
makes two HTTPS GETs plus a connection-details lookup; delete makes one HTTPS GET
and one POST. Both answer about the path given, unchanged.
```text theme={null}
databricks_action(
action="publish_designer_flow",
connection_id="my_databricks",
params={
"path": "/Shared/flows/runway_airport_mapping",
"pipeline": {
"steps": [
{"id": "read_runways", "kind": "read", "table": "main.raw.runways"},
{"id": "read_airports", "kind": "read", "table": "main.raw.airports"},
{"id": "join_airports", "kind": "join",
"inputs": ["read_runways", "read_airports"], "join_type": "inner",
"join_keys": [{"left": "airport_id", "right": "global_id"}]},
{"id": "write_mapping", "kind": "write", "inputs": ["join_airports"],
"table": "main.curated.runway_airport_mapping"}
]
}
}
)
```
#### **`delete_pipeline` cascade survivors**
Use `delete_pipeline` with `params.pipeline_id` and an explicit
`params.cascade`. When `params.cascade=true`, the action reads the pipeline's
target catalog and schema, inventories that schema before deletion, and then
sends the delete request. Metric views are read from the Unity Catalog REST
catalog. Tables whose names start with `__materialization_mat_` are read from
`information_schema.tables` using one SQL statement on a SQL warehouse. Pass
`params.warehouse_id` to select the warehouse explicitly; otherwise the action
uses the warehouse metadata stored with the connection. The warehouse must
already be `RUNNING`, because this action never starts a warehouse. The action
returns the delete result together with `target_catalog`, `target_schema`, and
the following survivor fields:
* `survivors_in_target_schema`: metric views (`table_type` is `METRIC_VIEW`) and
tables whose names start with `__materialization_mat_` that were present in the
target schema before deletion. These objects are named, not dropped. If no
survivor is known while either surface was unread or failed, this field is
`null`, not an empty list.
* `survivors_complete`: `true` only when both the Unity Catalog listing and the
`information_schema.tables` read reached their end; `false` when either surface
is incomplete or unavailable. An empty list is returned only after both reads
completed and found no survivor.
* `survivors_unknown_reason`: which survivor class could not be checked and why,
including when no SQL warehouse is available, the warehouse is not `RUNNING`,
or the SQL read fails.
The inventory is schema-wide and class-based, not a verified dependency check: an
unrelated metric view in the same schema is included, so confirm ownership before
acting on any listed object. A metric view whose source table the cascade removed
can keep resolving to nothing, causing dashboards and Genie agents to fail with
`UC_DEPENDENCY_DOES_NOT_EXIST` until the view is dropped or repointed. With
`params.cascade=false`, no schema inventory is performed and the survivor fields
are likewise unavailable. The REST listing costs one HTTPS request per page and
no warehouse compute; the materialization-table check costs one warehouse-state
request and, only for a `RUNNING` warehouse, one SQL statement whose compute is
billed for the statement's duration.
```text theme={null}
databricks_action(
action="delete_pipeline",
connection_id="my_databricks",
params={
"pipeline_id": "01234567-89ab-cdef-0123-456789abcdef",
"cascade": true,
"warehouse_id": "warehouse_id"
}
)
```
#### **Workspace transfer safety for `copy_workspace_object` and `move_workspace_object`**
| **Parameter** | **Applies to** | **Type** | **Default** | **Description** |
| - | - | - | - | - |
| **`connection_id`** | Both transfer actions | String | none | Source Databricks connection. |
| **`path`** | Both transfer actions | String | none | Absolute source Workspace path. |
| **`target_connection_id`** | Both transfer actions | String | none | Destination Databricks connection. |
| **`target_path`** | Both transfer actions | String | none | Absolute destination Workspace path. |
| **`overwrite`** | Both transfer actions | Boolean | `False` | Request replacement when the destination already contains the same object type. |
| **`confirm_overwrite`** | Both transfer actions | Boolean | `False` | Re-call after the user explicitly approves overwriting an existing destination object. |
| **`confirm_move`** | `move_workspace_object` only | Boolean | `False` | Re-call after the user explicitly approves deleting the source after a verified copy. |
| **`auto_create_directories`** | Both transfer actions | Boolean | `True` | Create missing parent directories on the destination workspace before import. |
If a transfer returns `requires_confirmation: true`, present the exact source
and destination connections and paths, the overwrite impact, and the
non-atomic move warning when applicable. Wait for an explicit user reply
before retrying with `confirm_overwrite=True` or `confirm_move=True`; never
self-confirm. If the destination already exists with a different object type,
choose a different `target_path` instead of overwriting.
`move_workspace_object` always requires `confirm_move=True` before deletion.
Moves are verified copies followed by best-effort source deletion and are not
atomic across workspaces. If a transfer is partial or indeterminate, retain
the source and inspect both paths rather than retrying automatically.
```python theme={null}
databricks_action(
action="copy_workspace_object",
connection_id="dev_databricks",
path="/Workspace/Shared/etl/transform_orders",
target_connection_id="prod_databricks",
target_path="/Workspace/Shared/etl/transform_orders",
)
```
```python theme={null}
databricks_action(
action="move_workspace_object",
connection_id="dev_databricks",
path="/Workspace/Shared/etl/transform_orders",
target_connection_id="prod_databricks",
target_path="/Workspace/Shared/etl/transform_orders",
confirm_move=True,
)
```
```python theme={null}
databricks_action(
action="sync_from_git",
repo_id="analytics_repo",
source_paths=["notebooks/analysis.py", "docs/README.md"],
target_workspace_path="/Workspace/Shared/project",
connection_id="conn_id",
)
```
#### **`edit_workspace_file`**
Use `edit_workspace_file` to change an existing Workspace `NOTEBOOK` or `FILE`
without sending the whole object through the tool. It reads and writes content
server-side using the selected Databricks connection. Supply the absolute
Workspace `path` either as the top-level argument or as `params.path`; if both
are supplied, they must name the same object. The action does not edit canonical
onboarding notebooks; use the guarded `create_notebook` action for those.
Choose exactly one edit mode:
| Parameter | Type | Default | Description |
| - | - | - | - |
| `params.old_text` | string | none | Required for a text-span edit; must be non-empty and occur exactly once, including whitespace and indentation. Add surrounding lines if the text is ambiguous. |
| `params.new_text` | string | none | Required with `old_text`; replacement text. Pass an empty string to delete the matched span. |
| `params.cell_index` | integer | none | Selects a notebook cell by its zero-based index. Cell edits apply only to notebooks. |
| `params.cell_op` | string | none | Notebook operation: `replace`, `insert_before`, `insert_after`, or `delete`. |
| `params.cell_source` | string | none | Source for `replace` and insert operations; required except for `delete`. |
| `params.expected_cell_sha256` | string | none | Required confirmation for a cell edit. First call without it to receive the selected cell's hash and preview; pass the returned hash on the edit call. A stale hash is refused. |
| `params.expected_sha256` | string | none | Optional whole-object concurrency guard. Use a previously returned `current_sha256` or `after_sha256`; the edit is refused if the object has changed. |
| `dry_run` or `params.dry_run` | boolean | `False` | Return the proposed diff without writing when `True`. |
An exact-span edit is refused if the text has zero or multiple matches. A cell
edit without `expected_cell_sha256` returns its current hash and a bounded cell
preview without writing. Notebooks keep at least one cell, so the last cell
cannot be deleted. Supported notebook languages are `PYTHON`, `SQL`, `SCALA`,
and `R`. When replacing or inserting a `%md` cell, include the `# MAGIC`
prefixes in `cell_source` as they appear in the notebook source.
The result includes before/after SHA-256 hashes, byte counts and delta, and a
bounded unified diff (`diff_truncated` indicates that the diff was cut). Applied
edits are exported again and report whether the stored content matches the
proposed text. Content itself is not returned. An applied edit makes four
Workspace API calls (status, export, import, and verifying export); a dry run
uses two (status and export).
```text theme={null}
databricks_action(
action="edit_workspace_file",
connection_id="my_databricks",
path="/Workspace/Shared/etl/transform_orders",
params={
"old_text": "WHERE status = 'pending'",
"new_text": "WHERE status = 'ready'",
"expected_sha256": ""
},
dry_run=True,
)
```
### `sync_from_git` path mapping
`sync_from_git` maps each selected source to the requested
`target_workspace_path` before uploading it. For a directory, the directory's
basename becomes the first path component and repository ancestors above that
directory are not reproduced. For example, selecting
`projects/acme/notebooks` with `target_workspace_path="/Workspace/Shared/project"`
deploys `projects/acme/notebooks/analysis.py` to
`/Workspace/Shared/project/notebooks/analysis` (the `.py` notebook-source
extension is removed). A single source file such as `projects/acme/job.py`
deploys directly to `/Workspace/Shared/project/job`. Set
`preserve_structure=False` to flatten all selected files to their basenames.
The response includes sorted `deployed_paths` with the exact workspace paths
accepted by Databricks. The success message lists up to 10 paths and reports
how many additional paths were deployed. If a computed path diverges from the
requested target, the file is not deployed and the response includes
`misplaced_files` with `success: false`. If different source files claim the
same destination, none of those contested destinations are deployed; the
response includes `colliding_files` and marks the sync unsuccessful.
```python theme={null}
databricks_action(
action="run_sql_batch",
params={
"warehouse_id": "warehouse_id",
"statements": [
"CREATE OR REPLACE TEMP VIEW recent_orders AS SELECT * FROM main.sales.orders",
"SELECT COUNT(*) AS order_count FROM recent_orders",
],
},
connection_id="conn_id",
)
```
## **Use Cases**
1. **Automated data processing**
Run ELT notebooks on a schedule and collect run logs for audit.
2. **ML training orchestration**
Kick off parameterized training runs and export resulting notebooks or metrics.
3. **Cross-environment promotion**
Copy or move existing Workspace notebooks and files between Databricks connections with content verification, or sync notebook source files and workspace assets from Git into `/Workspace/...` while preserving folder structure.
4. **Operational monitoring**
Poll execution status, capture stdout and error traces, and surface results to stakeholders.
5. **Batch SQL orchestration**
Submit related DDL or DML statements together with `run_sql_batch` when a workflow needs ordered execution.
**IMPORTANT:** Notebook execution can modify data and incur compute costs. Use non-production clusters for testing and set wait limits conservatively.
## **Workflow/How It Works**
1. **Authenticate**
Provide `connection_id`, or configure `DATABRICKS_HOST` and `DATABRICKS_TOKEN`.
Databricks Apps must use a Databricks connection: PAT connections and direct
bearer-token overrides are refused there; choose a **User OAuth** or
**Service Principal (OAuth M2M)** connection.
2. **Choose an action**
For example `copy_workspace_object`, `create_notebook`, `sync_from_git`, `upload_file`, `run_notebook`, or `run_sql_batch`.
3. **Provide context**
Set `path`, `target_connection_id`, `target_path`, `source_paths`, `target_workspace_path`, `file_path`, `params`, and other action-specific inputs as needed.
4. **Execute and observe**
Receive job or run identifiers, then fetch logs or results until completion.
5. **Promote or share**
Export artifacts, store with `file_manager_tools`, and link runs back to tasks or tickets.
Use consistent workspace paths like `/Workspace/Shared/project`. For an existing Workspace notebook or file on another connection, use `copy_workspace_object` or `move_workspace_object`. For a single new notebook, use `create_notebook`; for repo-backed files, use `sync_from_git`. `upload_file` is for DBFS and Unity Catalog Volume paths, not workspace paths.
## **Integration Relevance**
* **`data_connector_tools`**: Open connections and run SQL inside notebooks that the action executes.
* **`file_manager_tools`**: Capture notebook exports or logs and attach them to missions or tasks.
* **`project_manager_tools`**: Track notebook work as todos with linked runs and outcomes.
* **`git_action`**: Version notebook source, diffs, and promotion artifacts.
* **`dbt_action`**: Coordinate dbt transformations before or after a Databricks notebook run.
## **Configuration Details**
* Generate a Databricks personal access token with the permissions required for your operations.
* Prefer `connection_id` over direct host and token handling when possible.
* Ensure `cluster_id` points to a running or auto-start cluster that your token can access.
* Align `format` with your import or export destination.
* `sync_from_git` supports notebook source files `.py`, `.sql`, and `.scala`, plus workspace file types `.md`, `.csv`, `.json`, `.yaml`, `.yml`, `.txt`, `.toml`, `.cfg`, `.ini`, `.xml`, `.html`, and `.r`.
* For notebook source files, `sync_from_git` writes the Databricks notebook header with the file's language comment marker: `# Databricks notebook source` for Python, `-- Databricks notebook source` for SQL, and `// Databricks notebook source` for Scala. If a source file already has a header with a different marker, Genesis replaces it so the uploaded notebook has exactly one language-correct header.
* `bulk_upload` recognises Databricks notebook headers in `.py`, `.sql`, and `.scala` files uploaded outside `/Volumes/`. When the first line is a language-specific Databricks notebook header, the response includes a non-blocking warning recommending `sync_from_git`, which removes the source extension when importing notebooks.
* `bulk_upload` and `sync_from_git` never include `.git` metadata, regardless of include patterns. `sync_from_git` also refuses source paths outside the selected repository or paths that resolve into `.git`.
* `run_sql_batch` requires `params.statements` and accepts up to 50 SQL strings in one call.
* Parameter names in `params` must match widgets or parameter parsing in the notebook when you use notebook execution actions.
## **Limitations or Notes**
1. Requires valid workspace access and API permissions.
2. API rate limits and cluster capacity can throttle concurrent runs.
3. Large exports or logs may take time to transfer.
4. `sync_from_git` requires explicit `source_paths`; the repo root values `""`, `/`, `.`, and `./` are rejected.
5. `upload_file` does not support `/Workspace/...` or `/Users/...` destinations.
6. `copy_workspace_object` and `move_workspace_object` support only `NOTEBOOK` and `FILE` objects. Notebook languages must be `PYTHON`, `SQL`, `SCALA`, or `R`. Each object is limited to 10 MB for Workspace export/import. Directories, repos, libraries, and dashboards are not supported.
7. Some advanced workspace features are not exposed in REST endpoints.
8. Tokens expire and must be rotated according to your security policy.
### **Output**
* **Create and update**
Confirmation with notebook metadata and path.
* **Run execution**
Run or job identifier, life-cycle state, result summary, and log access.
* **Failed notebook runs**
By default, `check_run_status` and `run_notebook_live` return a bounded
`error_digest` containing the error class, exception type, message, SQLSTATE,
notebook cell and line, source line, and Spark suggestions, plus trimmed error
rows with the task, message, and traceback excerpt. Repeated task failures are
de-duplicated, and additional rows are counted in `errors_omitted`.
When trimming removes at least 4 KB, the untrimmed errors are saved and the
response includes `error_trace_spill_path` and guidance for reading the file
from the `apps` repository. If saving fails, the response explains that only
the digest and excerpt remain. Set `verbose_errors=True` to receive the raw
error payload, including the full JVM stack, Spark logical plan, and duplicated
task rows.
* **Listing**
Arrays of notebooks with paths, languages, and modification timestamps.
* **Import and export**
Content in the requested `format` and related download details. For
`get_notebook` and `export_notebook`, the response includes `language` and
`language_source`. `language` is the language reported by the export response
or workspace object metadata, and is `null` when neither source reports one;
it should not be assumed to be `PYTHON`. `language_source` is one of
`export response`, `workspace get-status`, `not reported`, or
`metadata unreadable`, so callers can distinguish an absent platform value
from a metadata lookup failure.
* **SOURCE notebook imports**
When a `create_notebook` import with `format="SOURCE"` may have malformed
separators, unprefixed magic directives, or empty cells, the response includes
a non-blocking `structural_warnings` array with human-readable line or cell
diagnostics.
* **`sync_from_git` validation**
Repository notebook syncs return per-file validation results and, when
applicable, `unvalidated_files`, `drifted_files`, `structure_lost_files`,
`unverified_files`, and aggregated `structural_warnings`. A sync with an
unvalidated file returns `success: false`.
* **Workspace transfer**
Source and destination metadata, exact-content validation (`validation` with SHA-256 and byte count), destination URL, and partial-move state such as `copy_completed`, `source_deleted`, and `source_retained`. Confirmation responses include `requires_confirmation`, `confirmation_fields`, and `suggested_next_calls`.
* **Pipeline deletion**
`delete_pipeline` returns `survivors_in_target_schema`, `survivors_complete`, and `survivors_unknown_reason` alongside the delete result when applicable. `published_tables_dropped` remains `null` because the action does not read the schema after deletion.
* **Errors**
HTTP status, Databricks error code, message, troubleshooting hints, and for invalid `action` values a suggested close match plus `valid_actions` when available.