Skip to main content

​
Overview

​
Tool Name

​
Purpose

The dbt_action tool lets you operate DBT projects programmatically. Use it to initialize projects, run and test models, compile SQL, install packages, generate documentation, and coordinate multi-environment deployments. Local dbt actions that would start dbt are refused inside a Databricks App: agent-authored Jinja, SQL, and Python models would run unconfined on the App host. Run the project as a dbt task in a Databricks Job instead. This restriction is specific to Databricks Apps; see the App deployment limits.

​
Functions Available

  1. dbt_action: Executes DBT operations for project management, model execution, testing, docs, snapshots, seeds, and dependency management. Controlled by the action parameter.

​
Key Features

Run and Test

Execute model runs and tests with targets, selectors, and parallel threads.

Compile and Document

Compile SQL for inspection and generate searchable documentation sites.

Packages and Artifacts

Install dependencies, seed data, snapshot state, and clean build outputs.

Multi-Env Targets

Switch profiles and targets for development, staging, and production.

Selective Execution

Use selectors to run only what you need for faster iteration.

​
Input Parameters for Each Function

​
dbt_action

Parameters

​
Repository dbt runtime

Pin a repository’s dbt runtime in .genesis/dbt-version when it needs a different dbt-core or adapter version than the Genesis installation. The file must contain dbt-core exactly once and may contain one exact pin per adapter, with one package pin per line. Blank lines and comments beginning with # are ignored.
Supported adapters are snowflake, duckdb, sqlite, postgres, bigquery, redshift, databricks, spark, trino, and clickhouse. Genesis builds an isolated environment on the first dbt-executing action for a new declaration; that build can take a few minutes. Later calls reuse the completed environment. The runtime cache keeps up to 10 completed repository runtimes and may remove the least-recently-used entries when another runtime is built. Every dbt-executing response, and list_system_profiles, includes a dbt_runtime object: Before invoking dbt, Genesis checks require-dbt-version in dbt_project.yml against the selected dbt-core version. If it does not match, the action returns a structured error and does not start dbt.

​
PLAN_MIGRATION and GENERATE_YAML

Use PLAN_MIGRATION to inspect a related set of views before converting them. It returns deterministic dependency order, dependencies, cycles, blocked references, detected sources and source-name collisions, parse failures, and a per-view relation_map; it does not return SQL text. Select the Git repository and dbt project with repo_id and project_name. Both actions accept exactly one locator: a list of repository SQL files, a repository directory, or a connection and fully qualified view names. Repository locators require source_dialect; for a connection locator, Genesis resolves the dialect from the connection when it is omitted. Listing a source directory searches recursively for .sql files. A connection locator performs one metadata query per view, and each located definition is parsed once per call. PLAN_MIGRATION and CONVERT_SQL resolve derived table references against the selected project’s source declarations and model files under its configured model paths. This reads dbt_project.yml and every YAML and .sql file under those paths; it does not use target/manifest.json. An explicit relation_map still takes precedence. A declared source replaces a derived source() only when its database, schema, and identifier are all statically known and match the relation. For a two-part relation, the view’s own database must also be known from the connection’s object_name or a Snowflake CREATE VIEW database.schema.view definition. Unproven declarations and same-named models are candidates for review, not automatic mappings; the derived call stays in place. Each relation_decisions entry reports the relation’s chosen call, basis, and possible candidates; the basis is declared_source, derived, needs_review, or relation_map. PLAN_MIGRATION substitutes a project’s source call into a view’s relation_map only when the declared source is proven to match. GENERATE_YAML accepts the same locators. It writes models/sources.yml and one models/<model>.yml schema skeleton per view in the selected project. It does not commit the files, and existing generated targets are not replaced unless overwrite=true. YAML generation requires every located view to parse successfully. Grain-like *_id and *_key columns are reported as candidate grain tests only; no tests are activated. When a case-insensitive source does not reveal the stored case needed by the target, sources_with_assumed_target_case lists the affected source identities for review. The response includes the final per-view relation_map; pass each view’s map unchanged to its CONVERT_SQL call so the generated models use the same source() and ref() mappings as sources.yml.

​
CONVERT_SQL

Use CONVERT_SQL to convert one Snowflake or SQL Server view definition into a generated dbt model. The source comes from either a repository SQL file or a live database view; provide exactly one locator. connection_id may accompany source_path as a source of warehouse column metadata, not as a second locator. Whenever a connection is supplied, conversion reads column names and types for base tables not covered by schema_path, using one bounded INFORMATION_SCHEMA.COLUMNS query per database and one more per additional 1,000 tables in that database. Each query applies a 50,000-row safety bound, with one extra row used to detect overflow. This metadata lets SELECT * and type expressions use the warehouse columns; if metadata cannot be read completely, the affected tables stay unknown and the response includes a reason. The connection must be Snowflake or SQL Server, and its dialect must match source_dialect. Conversion reads and writes repository files, but does not commit changes or return the source or generated SQL bodies. When conversion derives source() calls, it also reads dbt_project.yml and YAML and .sql files under the project’s model paths once per call to resolve project relations and check declarations. The generated file is <project>/models/<model_name>.sql. Select the repository and dbt project with repo_id and project_name. For example, convert a SQL file to a Databricks model:
To convert a repository SQL file and use Snowflake catalog columns for tables not listed in schema_path:
Genesis checks the transformed SQL’s structure before writing. A structurally_consistent verdict permits a write when no other write guard blocks it. requires_validation and not_checked results are marked with a validation header. A differs verdict or any unmapped relation blocks the write unless allow_unverified=true; an overridden write is marked for validation. An existing target file is not replaced unless overwrite=true. Sources larger than 8 MiB are rejected. Review the returned verdict, reasons, and next step before using the model. When connection_id is supplied, fetched_metadata_tables lists the tables whose column metadata was read. Tables with unavailable or incomplete metadata remain unknown and are named in reasons; review the verdict before using the model. For derived source() calls, Genesis checks whether the declared source points to the same database, schema, and identifier as the view; a known mismatch or an unproven project match prevents a structurally_consistent result and changes it to requires_validation, while other verdicts are not upgraded. An undeclared source is listed in undeclared_sources, with a proposed_sources_yml proposal for review; the YAML is never written to the repository. The proposal uses the target warehouse’s source-table spelling and quoting; sources_with_assumed_target_case lists source identities whose stored case could not be determined from a case-insensitive source and therefore needs review.

​
parse and compile

Use these project actions before run when you want dbt diagnostics without executing models:
  • parse reads the entire project and builds its manifest without a warehouse round-trip. It surfaces Jinja and YAML errors, unresolved ref() references, and duplicate model names. It always parses the whole project; selectors are not accepted.
  • compile generates the SQL dbt would run without executing a model. It can expose quoting, CTE-ordering, and macro-expansion issues that parsing does not. Pass model_name to compile one model; without it, the whole project is compiled. dbt opens a warehouse connection to resolve the target database and schema, but does not query table data.

​
Project Selection and Repository Layout

Project-scoped actions use the Git checkout identified by repo_id. Genesis discovers valid dbt projects at the repository root and in its immediate child directories; it does not recursively search deeper directories. Use action: list_projects to see each project’s canonical name, directory, profile, and the current selection. For actions that operate on an existing project, pass project_name using the canonical name declared in that project’s dbt_project.yml. For compatibility, an immediate child directory name is also accepted when it does not resolve to a different canonical project name. Use . to select a valid project at the repository root. Project selectors must be a single, trimmed name without path separators, a drive-colon, or ... action: create_project creates a new project in a child directory and rejects duplicate canonical names or an already-existing directory. A repository-root project can be selected and used, but action: delete_project cannot delete it.

​
test_profile

Tests a dbt profile connection without running any models. Use it to verify connectivity before starting dbt operations. The response includes the connection test result, profile name and location, whether the profile is a system profile, connection type and details, target, dbt debug output, and a success or failure message. Host paths in the profile location and debug output are returned in redacted or relative form. Connection diagnostics are available when secret fields such as password, token, private_key, and keyfile contain redaction placeholders such as ***REDACTED***; diagnostics containing real secret values are blocked. Safe diagnostic states such as is required, none configured for profile default, and not set remain available, including when they appear in structured dbt output; credential-like values remain blocked.
Use select and exclude to scope execution for quick feedback during development.
Genesis validates action-specific parameters before execution. If an action does not accept an explicitly provided parameter, the tool returns success: false with error: "Invalid parameters for dbt action '<action>'.", error_class: "tool_error", and error_type: "TypeError" instead of starting the dbt action.
query and sql_content support normal dbt/Jinja template syntax, including patterns such as ref(), source(), var(), config(), is_incremental(), log(), adapter dispatch, and ordinary control flow. For log(), the request judge allows only literal message arguments and any supplied info argument must be a literal Boolean; the message can contain no secret, credential, environment, host, account, or absolute-path data. Dynamic expressions, concatenations, variables, macros, attributes, and sensitive literal content remain deny cases. Genesis only runs the Jinja-specific dbt safety checks when the submitted text actually contains dbt/Jinja delimiters such as {{ ... }} or {% ... %}.
The dbt judge is instructed to respond with exactly ALLOW or DENY: <reason_code>, without a free-form rationale. Category extraction tolerates letter case, surrounding whitespace, and a sole code fence already accepted by the verdict parser; no user- or model-authored prose is returned. A denial keeps block_reason_code: "judge_denied" and adds judge_reason_code with one of secret_exposure, environment_access, host_details, file_or_process_access, protected_content, sandbox_bypass, or other_policy. Legacy free-form, missing, unknown, inline-decorated, extra-content, duplicate, or conflicting categories use unspecified and the generic denial summary. Genesis selects all public summaries from fixed text and never reflects the raw judge response or rejected dbt input to the caller.

​
Use Cases

  1. Automated Transformations Schedule run in CI to keep curated models fresh.
  2. Data Quality Gates Execute test on pull requests and block merges on failure.
  3. Documentation Portals Generate docs and publish the site for analysts and stakeholders.
  4. Incremental Model Hygiene Trigger full_refresh during backfills or schema changes.
  5. Dependency Management Pin versions and run deps to ensure reproducible builds.

​
Workflow/How It Works

  1. Step 1: Discover or Create a Project Use action: list_projects to inspect the repository, action: use_project with project_name to select an existing project, or action: create_project with a new child-directory project_name to create one.
  2. Step 2: Configure Profiles and Targets Provide profiles_dir and target for warehouse connections.
  3. Step 3: Install Packages Run deps to install packages from packages.yml.
  4. Step 4: Run and Test Execute run with select, models, or model_name, then test with the same model_name to validate that model’s tests.
  5. Step 5: Compile and Document Use compile to inspect SQL, then docs to build the catalog and site.
  6. Step 6: Operate and Maintain Use seed, snapshot, and clean as needed for datasets and state.

​
Get a Fresh Manifest

Use action: get_manifest to retrieve the project’s compiled dbt graph. By default, Genesis returns the existing target/manifest.json without running dbt, so it may predate files written in the current working tree. Set force_refresh: true whenever you use get_manifest to check that your latest changes parse; Genesis deletes the cached artifact and recompiles the project. Every response includes cache_hit, which is true when an existing artifact was returned without re-running dbt, and generated_at, taken from the manifest’s metadata.generated_at value or null when that value is absent. If compilation fails, the response sets success: false and includes dbt’s own error output so the failure is actionable.
Process isolation. Every dbt CLI parse or execution runs in a subprocess. This keeps Jinja, adapter, and package Python outside the Genesis server process, avoids blocking the shared asyncio event loop, and ensures file-backed adapter handles die when the child exits. Genesis reads the profile as safe YAML only to construct the restricted child environment.Parallel subprocess invocations. When Genesis runs dbt in a subprocess each child receives a minimal invocation-owned environment and a private profiles.yml copy (directory mode 0700, file mode 0600). The child does not inherit arbitrary Genesis server variables. The private file contains only the selected profile and target, so unrelated credentials never enter the child. Exact whole-value Snowflake credential references in supported profile fields are materialized safely; dynamic, concatenated, defaulted, filtered, nested, or unknown profile env_var(...) expressions fail closed. Calls within one repository serialize because dbt shares project files and target/ artifacts; different repositories remain concurrent without cross-contaminating profile resolution or credentials. Genesis bounds how many dbt subprocesses run at once with dbt.max_concurrent_invocations (default 8, minimum clamped to 2 at runtime) to limit warehouse contention without serializing dbt to one-at-a-time; changing the setting requires a process restart.Each project-scoped invocation carries an invocation-owned project directory resolved from the repository snapshot. For untrusted actions that parse dbt project files, Genesis inventories the selected project’s parse inputs before dispatch and checks that the project and inventory are unchanged before the subprocess starts. If the project changes during this safety preflight, the action fails and should be retried.

​
Integration Relevance

  • data_connector_tools for connectivity checks and schema exploration.
  • git_action to version models, macros, and tests.
  • file_manager_tools to store artifacts and publish generated docs.
  • project_manager_tools to track DBT tasks and milestones.
  • databricks_action when models rely on Databricks execution.

​
Configuration Details

  • Ensure profiles.yml contains correct credentials for each target.
  • When using setup_profile, do not put direct env_VAR or ${VAR} references in credential fields. Genesis rejects those references instead of resolving them from the server environment; use a stored secret or the isolated env_var(...) profile flow described below.
  • Tune dbt.max_concurrent_invocations (default 8, env var GENESIS_DBT_MAX_CONCURRENT_INVOCATIONS) when many agents run dbt builds in different repositories — it caps concurrent subprocesses to limit warehouse contention (values below 2 are clamped up at runtime). Calls in the same repository serialize around their shared files. Restart Genesis after changing this setting.
  • Tune threads based on warehouse capacity and model complexity.
  • For Snowflake query attribution and custom dbt query_tag values, see Identify Twin scanner queries.
  • Pin package versions in packages.yml for reproducible builds.
  • For trusted automation that needs non-secret env_var(...) values, configure exact name/value pairs in dbt.safety.trusted_env_vars. Server process environment variables are never inherited; config-side env_VAR and ${VAR} references are rejected rather than resolved, and runtime-control names are reserved.
  • For file-backed DuckDB targets, point profiles.yml at the same .duckdb file Genesis uses for agent queries only when both workloads can share the database; each dbt child releases its file handle when it exits, but a concurrent dbt child still needs the file unheld for the duration of its run.
DBT runs execute SQL on your warehouse. Choose the correct target, set threads responsibly, and use full_refresh only when necessary.
Genesis applies dbt input and output safety checks around execution unless the project is on the dbt.safety.trusted_projects allowlist (see Advanced Settings — dbt safety settings). A trusted match on project_name or repo_id skips the static blocks and LLM judge; it does not inherit the server environment. Only non-secret values explicitly configured in dbt.safety.trusted_env_vars are available to the trusted project’s env_var(...) calls. Host-path and credential redaction in dbt_action always still runs. Trusted project-name matches require one unique canonical project name within the repository; duplicate names fail closed rather than bypassing the safety checks. Untrusted projects: literal secret indicators or sensitive host paths in query, sql_content, config_updates, or dbt project files under models/, macros/, tests/, snapshots/, or analyses/ can be blocked even when the input is plain SQL with no Jinja. Inputs or returned logs that try to expose env vars, host files, session tokens, bot_storage, AWS_SECRET_ACCESS_KEY, PRIVATE KEY, or other exposed credential values can fail with a generic safety error. Caller-supplied credentials are excluded from the return-side safety judge’s request summary and from dbt diagnostic error context. If a configured dbt credential cannot be resolved, the tool returns a generic credential-resolution error with error_type: "SecretResolutionError" rather than exposing the resolver’s exception details. Redacted, masked, hidden, removed, omitted, scrubbed, or sanitized placeholder values are allowed in returned output. The safety judge accepts a verdict when ALLOW or DENY opens a line, including common Markdown or code-fence decoration and a trailing separator such as :, -, –, —, or .. A line-start DENY always blocks; an ALLOW is accepted only when no DENY appears anywhere in the raw verdict. Verdicts with only preamble or prose, and other ambiguous or unparseable verdicts, fail closed with block_reason_code: "judge_unparseable". The raw judge verdict is retained in the structured safety audit record for operators, but is not returned to the caller.

​
Limitations or Notes

  1. Requires DBT CLI availability in the execution environment.
  2. Large graphs can consume significant compute, scope with select.
  3. Adapter features vary by warehouse; validate per adapter.
  4. Network and warehouse quotas can impact execution times.

​
Output

  • Command Execution: Logs with status, timings, and summary counts.
  • Model Results: Per model status and row counts.
  • Test Results: Pass or fail with failure details.
  • Docs and Compile: Generated site files and compiled SQL outputs.
  • Runtime: dbt-executing responses include dbt_runtime; see Repository dbt runtime for the declaration format, response fields, and version preflight.
  • Errors: On failure, responses set success: false and populate error from the tail of captured dbt output when the runner leaves it empty — so agents always get an actionable summary. Check error_class to distinguish a genuine dbt or dispatch failure (tool_error) from a safety policy block (policy_block); the tool-call endpoint reference documents the associated block reason fields. Full output may also appear in logs (or a spill-to-file hint when inline size is capped); read spilled logs with git_action(action='read_file', ...). get_manifest also returns dbt’s compile error output instead of a generic manifest-generation error. Credential-resolution failures use a fixed generic message and error_type: "SecretResolutionError"; credential values and resolver exception details are not included in the response context.