Databricks Action Tools

​

Overview

​

Tool Name

databricks_action

​

Purpose

​

Functions Available

1. databricks_action: Performs Databricks operations, including notebook lifecycle actions, workspace file sync, SQL execution, job control, and status checks through the Databricks APIs.

​

Key Features

​

Input Parameters for Each Function

​

databricks_action

​

Action Routing

​

In-thread completion notices

​

update_job task replacement and persistence checks

databricks_action(
  action="update_job",
  job_id=123,
  params={
    "mode": "merge",
    "tasks": [{"task_key": "daily_sales", "notebook_task": {"base_parameters": {"date": "today"}}}],
  },
)

databricks_action(
  action="update_job",
  job_id=123,
  params={
    "mode": "replace",
    "tasks": [],
    "include_persisted_definition": True,
  },
)

​

list_clusters compute scope

• Each cluster includes cluster_source. When Databricks does not provide that field, the tool returns "UNKNOWN"; do not treat it as an all-purpose cluster.
• serverless_compute.status is "unknown" because this API does not report whether serverless is enabled. An empty or all-terminated cluster list does not mean that notebooks cannot run.
• For serverless job compute, call run_notebook or run_notebook_live without cluster_id. For run_notebook, passing the literal cluster_id="serverless" selects the same serverless Jobs path; do so only when serverless execution has been explicitly approved. Use list_jobs and check_run_status to see whether runs have already succeeded through that path.
• SQL warehouses are a separate compute path listed by get_workspace_info. They can serve SQL but cannot run a notebook. Check these paths before reporting that no compute is available.

​

execute_python process isolation and recovery

• Use connection_id to select the Databricks workspace. execute_python uses serverless compute automatically; do not pass cluster_id for this action.
• Use session_id to preserve variables between calls. If it is omitted, the response creates a session and returns its session_id. Sessions expire after 10 minutes without use; check session_expires_at or use list_python_sessions to inspect the current thread’s sessions.
• When a new session is created because session_id was omitted, was not found, or had expired after more than 10 minutes of inactivity, the response includes session_note. This is normal session recycling, not a context limit or an error. If the previous session expired or the supplied ID was missing, rerun the setup code; when no ID was supplied, save the returned session_id for subsequent calls.
• A normal Python exception is returned in the result with success: false, error, error_type, and traceback details. A crashed execution process is returned with error_type: "SparkWorkerCrashed" and lost_sessions; every session listed there has lost its variables.
• If the worker fails during startup, its captured output may include spark worker entrypoint reached pid=.... This means the child reached its entrypoint; if the marker is absent, the process may have failed before then, so investigate worker startup or spawning.
• After SparkWorkerCrashed, omit session_id to create a fresh session and rerun the setup code. The response explains that the Genesis server was unaffected; do not treat this result as a server outage.

databricks_action(
    action="execute_python",
    code="result = spark.sql('SELECT COUNT(*) FROM catalog.schema.table').collect()",
    connection_id="conn_id",
)

​

Command execution and confirmation

databricks_action(
    action="execute_command",
    cluster_id="cluster_id",
    language="PYTHON",
    command="spark.sql('INSERT INTO catalog.schema.table SELECT * FROM catalog.schema.staging')",
    params={"confirm": True},
    wait_for_result=False,
)

​

Stopping remote Databricks work

• run_notebook_live cancels the Databricks job run identified by databricks_run_id.
• run_notebook_cells and execute_command cancel the active command on its cluster using cluster_id, context_id, and command_id.
• run_sql_statement cancels the SQL Statement API request it is polling. This also applies when a spec-author seat’s hard-stop stops the tool call. The statement cancellation is best effort; the tool reports the stop even if Databricks does not confirm cancellation.
• Genesis retries the remote cancellation once after a one-second delay. The tool still reports the user stop even if both cancellation attempts fail.
• For execute_command with wait_for_result, Genesis destroys the temporary execution context after the stop, including when cancellation fails.

​

Uncertain notebook submission

​

SOURCE notebook structure warnings

• A cell separator that is not exactly the language-specific marker on its own line, such as # COMMAND ---------- for Python or -- COMMAND ---------- for SQL.
• A Databricks magic directive such as %md, %sql, or %python that is not prefixed with the language-specific MAGIC marker.
• An empty cell caused by a lost or duplicated cell separator.

• VERIFIED: The stored cells match the source, or the notebook has no Markdown cell that could cause this type of drift.
• DRIFTED: An intended code cell was stored as Markdown. The path is also listed in drifted_files; this is the only result that establishes the code will not execute, even if a job reports SUCCESS.
• STRUCTURE_LOST: Databricks stored a different number of cells than the source sent. The path is listed in structure_lost_files, and which cells execute is not established.
• UNVERIFIED: The JUPYTER export check could not run. The path is listed in unverified_files, and executability is unknown rather than known-bad.

​

stage_from_s3 AWS credentials

​

SQL Safety for run_sql_statement and run_sql_batch

​

Unity Catalog metric views

databricks_action(
  action="list_metric_views",
  connection_id="my_databricks",
  params={"catalog": "main", "schema": "analytics"}
)

​

Unity Catalog registration for the semantic layer

databricks_action(
  action="register_uc_assets",
  connection_id="my_databricks",
  params={
    "catalog": "main",
    "schema": "analytics",
    "dry_run": true,
    "assets": [
      {
        "table": "orders",
        "comment": "Customer orders",
        "columns": {"order_id": "Unique order identifier"},
        "provenance": {"source_system": "snowflake", "source_path": "SALES.ORDERS"}
      }
    ]
  }
)

​

Databricks Genie spaces and semantic-layer binding

databricks_action(
  action="bind_semantic_layer_to_genie",
  connection_id="my_databricks",
  params={
    "catalog": "main",
    "schema": "analytics",
    "only_metric_views": true,
    "verify_question": "Which carrier has the worst on-time performance?"
  }
)

​

Unity Catalog domains and governed tag policies

• list_tag_policies takes no action-specific parameters and returns the account-level policies visible to the identity with one HTTPS request.
• create_tag_policy requires params.tag_key; optional params.description and params.allowed_values[] define the policy; params.dry_run defaults to false. Creating a policy writes an account-level object, not a workspace-scoped setting. It makes one HTTPS POST and, when allowed values are supplied, one PATCH to store them; inspect allowed_values_stored before assigning values.
• create_domain requires params.tag_key for an existing policy. Optional params.subtitle, params.description, params.parent_domain_id and params.icon are sent with the domain; params.dry_run defaults to false. It lists domains and then makes one HTTPS POST only when no domain exists on that key. A listing error prevents creation.
• list_domains takes optional params.parent_domain_id to filter visible domains and makes one HTTPS GET. A non-empty result carries visibility_limited: false and reports the visible count. An empty result carries visibility_limited: true and says that this identity cannot see any domains, while some may exist; when filtering by parent, the message names that parent. It does not prove none exist.
• assign_to_domain requires params.tag_key, params.value (one allowed by its policy), and params.assets[]. Each asset has catalog, schema, and optional table; omitting table assigns the schema. params.dry_run defaults to false and returns statements without running them. params.warehouse_id optionally selects the SQL warehouse; otherwise the connection’s configured warehouse is used.

​

Unity Catalog Pages

databricks_action(
  action="bulk_create_uc_pages",
  connection_id="my_databricks",
  params={
    "domain_id": "domain-id-from-list_domains",
    "dry_run": true,
    "pages": [
      {"name": "Net Revenue", "description": "Revenue after refunds", "synonyms": ["net sales"]}
    ]
  }
)

​

Unity Catalog external metadata and lineage

databricks_action(
  action="publish_context_subgraph",
  connection_id="my_databricks",
  params={
    "name_prefix": "migration-2026",
    "dry_run": true,
    "nodes": [
      {"qualified_name": "snowflake://SALES.ORDERS", "columns": ["ORDER_ID"]}
    ],
    "edges": []
  }
)

​

Lakeflow Designer visual flows

databricks_action(
  action="publish_designer_flow",
  connection_id="my_databricks",
  params={
    "path": "/Shared/flows/runway_airport_mapping",
    "pipeline": {
      "steps": [
        {"id": "read_runways", "kind": "read", "table": "main.raw.runways"},
        {"id": "read_airports", "kind": "read", "table": "main.raw.airports"},
        {"id": "join_airports", "kind": "join",
         "inputs": ["read_runways", "read_airports"], "join_type": "inner",
         "join_keys": [{"left": "airport_id", "right": "global_id"}]},
        {"id": "write_mapping", "kind": "write", "inputs": ["join_airports"],
         "table": "main.curated.runway_airport_mapping"}
      ]
    }
  }
)

​

delete_pipeline cascade survivors

• survivors_in_target_schema: metric views (table_type is METRIC_VIEW) and tables whose names start with __materialization_mat_ that were present in the target schema before deletion. These objects are named, not dropped. If no survivor is known while either surface was unread or failed, this field is null, not an empty list.
• survivors_complete: true only when both the Unity Catalog listing and the information_schema.tables read reached their end; false when either surface is incomplete or unavailable. An empty list is returned only after both reads completed and found no survivor.
• survivors_unknown_reason: which survivor class could not be checked and why, including when no SQL warehouse is available, the warehouse is not RUNNING, or the SQL read fails.

databricks_action(
  action="delete_pipeline",
  connection_id="my_databricks",
  params={
    "pipeline_id": "01234567-89ab-cdef-0123-456789abcdef",
    "cascade": true,
    "warehouse_id": "warehouse_id"
  }
)

​

Workspace transfer safety for copy_workspace_object and move_workspace_object

databricks_action(
    action="copy_workspace_object",
    connection_id="dev_databricks",
    path="/Workspace/Shared/etl/transform_orders",
    target_connection_id="prod_databricks",
    target_path="/Workspace/Shared/etl/transform_orders",
)

databricks_action(
    action="move_workspace_object",
    connection_id="dev_databricks",
    path="/Workspace/Shared/etl/transform_orders",
    target_connection_id="prod_databricks",
    target_path="/Workspace/Shared/etl/transform_orders",
    confirm_move=True,
)

databricks_action(
    action="sync_from_git",
    repo_id="analytics_repo",
    source_paths=["notebooks/analysis.py", "docs/README.md"],
    target_workspace_path="/Workspace/Shared/project",
    connection_id="conn_id",
)

​

edit_workspace_file

databricks_action(
  action="edit_workspace_file",
  connection_id="my_databricks",
  path="/Workspace/Shared/etl/transform_orders",
  params={
    "old_text": "WHERE status = 'pending'",
    "new_text": "WHERE status = 'ready'",
    "expected_sha256": "<before_sha256 from a previous read>"
  },
  dry_run=True,
)

​

sync_from_git path mapping

databricks_action(
    action="run_sql_batch",
    params={
        "warehouse_id": "warehouse_id",
        "statements": [
            "CREATE OR REPLACE TEMP VIEW recent_orders AS SELECT * FROM main.sales.orders",
            "SELECT COUNT(*) AS order_count FROM recent_orders",
        ],
    },
    connection_id="conn_id",
)

​

Use Cases

1. Automated data processing Run ELT notebooks on a schedule and collect run logs for audit.
2. ML training orchestration Kick off parameterized training runs and export resulting notebooks or metrics.
3. Cross-environment promotion Copy or move existing Workspace notebooks and files between Databricks connections with content verification, or sync notebook source files and workspace assets from Git into /Workspace/... while preserving folder structure.
4. Operational monitoring Poll execution status, capture stdout and error traces, and surface results to stakeholders.
5. Batch SQL orchestration Submit related DDL or DML statements together with run_sql_batch when a workflow needs ordered execution.

​

Workflow/How It Works

1. Authenticate Provide connection_id, or configure DATABRICKS_HOST and DATABRICKS_TOKEN. Databricks Apps must use a Databricks connection: PAT connections and direct bearer-token overrides are refused there; choose a User OAuth or Service Principal (OAuth M2M) connection.
2. Choose an action For example copy_workspace_object, create_notebook, sync_from_git, upload_file, run_notebook, or run_sql_batch.
3. Provide context Set path, target_connection_id, target_path, source_paths, target_workspace_path, file_path, params, and other action-specific inputs as needed.
4. Execute and observe Receive job or run identifiers, then fetch logs or results until completion.
5. Promote or share Export artifacts, store with file_manager_tools, and link runs back to tasks or tickets.

​

Integration Relevance

• data_connector_tools: Open connections and run SQL inside notebooks that the action executes.
• file_manager_tools: Capture notebook exports or logs and attach them to missions or tasks.
• project_manager_tools: Track notebook work as todos with linked runs and outcomes.
• git_action: Version notebook source, diffs, and promotion artifacts.
• dbt_action: Coordinate dbt transformations before or after a Databricks notebook run.

​

Configuration Details

• Generate a Databricks personal access token with the permissions required for your operations.
• Prefer connection_id over direct host and token handling when possible.
• Ensure cluster_id points to a running or auto-start cluster that your token can access.
• Align format with your import or export destination.
• sync_from_git supports notebook source files .py, .sql, and .scala, plus workspace file types .md, .csv, .json, .yaml, .yml, .txt, .toml, .cfg, .ini, .xml, .html, and .r.
• For notebook source files, sync_from_git writes the Databricks notebook header with the file’s language comment marker: # Databricks notebook source for Python, -- Databricks notebook source for SQL, and // Databricks notebook source for Scala. If a source file already has a header with a different marker, Genesis replaces it so the uploaded notebook has exactly one language-correct header.
• bulk_upload recognises Databricks notebook headers in .py, .sql, and .scala files uploaded outside /Volumes/. When the first line is a language-specific Databricks notebook header, the response includes a non-blocking warning recommending sync_from_git, which removes the source extension when importing notebooks.
• bulk_upload and sync_from_git never include .git metadata, regardless of include patterns. sync_from_git also refuses source paths outside the selected repository or paths that resolve into .git.
• run_sql_batch requires params.statements and accepts up to 50 SQL strings in one call.
• Parameter names in params must match widgets or parameter parsing in the notebook when you use notebook execution actions.

​

Limitations or Notes

1. Requires valid workspace access and API permissions.
2. API rate limits and cluster capacity can throttle concurrent runs.
3. Large exports or logs may take time to transfer.
4. sync_from_git requires explicit source_paths; the repo root values "", /, ., and ./ are rejected.
5. upload_file does not support /Workspace/... or /Users/... destinations.
6. copy_workspace_object and move_workspace_object support only NOTEBOOK and FILE objects. Notebook languages must be PYTHON, SQL, SCALA, or R. Each object is limited to 10 MB for Workspace export/import. Directories, repos, libraries, and dashboards are not supported.
7. Some advanced workspace features are not exposed in REST endpoints.
8. Tokens expire and must be rotated according to your security policy.

​

Output

• Create and update Confirmation with notebook metadata and path.
• Run execution Run or job identifier, life-cycle state, result summary, and log access.
• Failed notebook runs By default, check_run_status and run_notebook_live return a bounded error_digest containing the error class, exception type, message, SQLSTATE, notebook cell and line, source line, and Spark suggestions, plus trimmed error rows with the task, message, and traceback excerpt. Repeated task failures are de-duplicated, and additional rows are counted in errors_omitted. When trimming removes at least 4 KB, the untrimmed errors are saved and the response includes error_trace_spill_path and guidance for reading the file from the apps repository. If saving fails, the response explains that only the digest and excerpt remain. Set verbose_errors=True to receive the raw error payload, including the full JVM stack, Spark logical plan, and duplicated task rows.
• Listing Arrays of notebooks with paths, languages, and modification timestamps.
• Import and export Content in the requested format and related download details. For get_notebook and export_notebook, the response includes language and language_source. language is the language reported by the export response or workspace object metadata, and is null when neither source reports one; it should not be assumed to be PYTHON. language_source is one of export response, workspace get-status, not reported, or metadata unreadable, so callers can distinguish an absent platform value from a metadata lookup failure.
• SOURCE notebook imports When a create_notebook import with format="SOURCE" may have malformed separators, unprefixed magic directives, or empty cells, the response includes a non-blocking structural_warnings array with human-readable line or cell diagnostics.
• sync_from_git validation Repository notebook syncs return per-file validation results and, when applicable, unvalidated_files, drifted_files, structure_lost_files, unverified_files, and aggregated structural_warnings. A sync with an unvalidated file returns success: false.
• Workspace transfer Source and destination metadata, exact-content validation (validation with SHA-256 and byte count), destination URL, and partial-move state such as copy_completed, source_deleted, and source_retained. Confirmation responses include requires_confirmation, confirmation_fields, and suggested_next_calls.
• Pipeline deletion delete_pipeline returns survivors_in_target_schema, survivors_complete, and survivors_unknown_reason alongside the delete result when applicable. published_tables_dropped remains null because the action does not read the schema after deletion.
• Errors HTTP status, Databricks error code, message, troubleshooting hints, and for invalid action values a suggested close match plus valid_actions when available.