Skip to main content

​
Overview

​
Tool Name

​
Purpose

The talend_tools group lets Genesis Data Agents work with Talend Open Studio jobs without driving the Studio GUI. Genesis parses a project’s .item files to list and describe job structure (components, schemas, data flow, lineage summary, and observed migration evidence), and executes already-built standalone job artifacts through their <job>_run.sh launchers. Talend Open Studio has no headless API, so Genesis targets the two surfaces that are scriptable offline: project .item/.properties metadata and built job exports. Configure paths in Config -> Advanced Settings under the talend section (or via environment variables). See Talend setup.
talend_tools does not use Config -> Connections. All settings come from APP_CONFIG (talend.project_path, talend.jobs_dir, talend.runner_url, and related fields).

​
Configuration summary

When talend.runner_url is set, list_talend_jobs reports jobs_dir as the runner URL and lists built jobs from the remote service. describe_talend_job still reads local .item files and requires talend.project_path.

​
Function Reference

Genesis injects bot_id, user_id, and thread_id automatically. The tables below list only the user-supplied parameters.

​
Discovery and inspection

​
Execution

​
Return shapes

list_talend_jobs (success):
  • project_jobs — job names parsed from process/*.item files (inspect with describe_talend_job).
  • built_jobs — array of {name, directory} for artifacts run_talend_job can execute.
describe_talend_job (success):
  • job — name, job_type, context, context_params, components (with role: input, output, or transform), connections, and lineage (inputs / outputs).
  • Each component may also include migration evidence from the .item XML: table_values for nested TABLE cells, rules for observed tMap mapping or filter expressions and BigQuery SQL, and external_references for literal stored-procedure names found in BigQuery SQL. PostgreSQL Input and Row components may also include rctl_table_dependencies for static RCTL table names found in their SQL query parameters (QUERY, SQL, SQL_QUERY, or QUERY_TEXT). Each dependency includes its name, XML source location, and rows_status of unknown; the SQL and table rows are not returned. Procedure bodies are not available.
  • Evidence expressions and context defaults are redacted as [REDACTED] when their names or contents match credential-sensitive patterns. Observed expressions are evidence from the source job, not an approved migration specification.
run_talend_job (success):
  • run_id, job_name, status ("running"), and a message indicating whether Genesis is watching the remote run. When watched, Genesis wakes the originating thread with the run result after the runner reports succeeded, failed, timeout, or error; no follow-up get_talend_run call is needed. If the run cannot be watched, the message directs the agent to poll get_talend_run.
get_talend_run (success):
  • status — running, succeeded, failed, timeout, or error.
  • When terminal: exit_code, duration_seconds, timed_out, stdout, stderr (stdout/stderr are tail-truncated to 20,000 characters).

​
Usage Examples

​
Notes and limitations

  • run_talend_job is refused in a Databricks App, including when talend.runner_url points to a remote runner. See Talend setup.
  • Jobs must be built in Talend Studio before run_talend_job can execute them. Genesis runs the standalone <job>_run.sh launcher (a java -cp ... invocation that needs only a JDK).
  • For local runs on Linux, Genesis stops processes the job leaves behind when it exits, times out, or is cancelled, including processes detached with setsid or a double fork. It sends SIGTERM, waits up to 5 seconds, then sends SIGKILL if needed. This cleanup behavior does not describe remote talend-runner execution.
  • describe_talend_job parses Talend 8 DI .item XML locally. It does not call Talend Studio or Talend Cloud.
  • Run state is in-memory. If the Genesis server restarts, earlier run_id values may no longer resolve.
  • If Genesis cannot hand the job to its serving loop, run_talend_job marks the stored run error with a completion timestamp before returning the scheduling error. Do not keep polling that run as if it were still running.
  • For scheduler-driven orchestration of non-Talend jobs, use JAMS tools or Airflow Tools instead.
  • Twin’s Talend lineage scanner reuses the same .item parser for Context Graph ingestion. See Digital Twin and Context Graph.

​
Troubleshooting

  • Confirm talend.project_path points at a Talend project with process/*.item files.
  • Confirm talend.jobs_dir contains unzipped Build Job exports, each with a <job>_run.sh script.
  • When using a remote runner, verify talend.runner_url is reachable and returns jobs from GET /jobs.
  • Call list_talend_jobs and report the resolved project_path and jobs_dir from the response.
  • Set talend.project_path to a checkout of the Talend project on the Genesis host.
  • The remote talend-runner lists and runs built jobs only; it does not expose .item internals.
  • Use the exact job_name from list_talend_jobs → built_jobs.
  • Rebuild the job in Talend Studio and export it to talend.jobs_dir (or the runner’s jobs directory).
  • Poll the same Genesis instance that started the run.
  • Run state is not persisted across server restarts — submit the job again if needed.
  • Pass a JSON object string with string values, for example {"key": "value"}.
  • Genesis maps each key/value to Talend’s --context_param key=value launcher flags.