Skip to main content

​
Overview

​
Tool Name

​
Purpose

The image_tools group provides AI vision and image generation capabilities. You can ask natural language questions about images, create new images from prompts, and locate or retrieve image files that live in your workspace.

​
Key Features & Functions

Visual Analysis

Ask questions about charts, documents, UI screenshots, and scenes to extract text, structure, and context.

Text-to-Image

Generate illustrations, diagrams, and mockups from concise prompts with optional constraints.

File Management

Find and fetch images by name or path for reuse in workflows.

Batch Friendly

Process multiple images in sequence for consistent analysis at scale.

​
Input Parameters for Each Function

​
image_analysis

Parameters
Be explicit in your questions.
Example: “What is the Q2 total in the bottom-right bar and what color is that bar?”

​
image_generation

Parameters See Cost accounting for image-request metering. On a Databricks App with durable chat uploads enabled, an image generated for a conversation is kept across restarts when it is saved in that conversation’s uploads/ folder. If it cannot be kept, the result includes durability_warning when the volume cannot store it or the conversation refuses its attachment; the image remains on the current server only.

​
find_image_path_in_download_folder

Parameters

​
Use Cases

  1. Chart understanding Extract totals, trends, or anomalies from screenshots of dashboards.
  2. Document capture Read invoices, forms, screenshots, and other image-based documents to pull text and key fields.
  3. Asset creation Generate icons, diagrams, or hero images for reports and presentations.
  4. Product review Identify defects or features across batches of product photos.
  5. UX analysis Describe layout, hierarchy, and accessibility concerns in UI screenshots.
Check license and rights before generating or distributing images that resemble branded assets, people, or copyrighted material.

​
Workflow/How It Works

  1. Provide an image or prompt For analysis, supply file_name, optionally repo_id, and a clear question. For generation, write a concise prompt with style or size hints.
  2. Run the function The tool routes to a vision or image model, processes your request, and returns structured text or a new image artifact.
  3. Store and reuse Register generated images with your file manager and attach them to tasks or projects. Resolve stored image paths later by name when another workflow needs them.
Use file_name plus optional repo_id; Genesis resolves the stored image path for you. When repo_id points to a managed remote repo, Genesis uses the active WIP worktree when available; otherwise it reads the current remote file contents instead of a shared local clone. For PDFs, use the git_action OCR or visual page analysis workflows instead of image_analysis.

​
Integration Relevance

  • file_manager_tools to register, attach, and share images with entities.
  • artifact_manager_tools to persist and link generated outputs.
  • project_manager_tools to track visual assets by mission or task.
  • document_index_tools to index extracted text from image-based documents.
  • git_action for versioned storage of static image assets.

​
Configuration Details

  • Use supported image formats such as PNG, JPEG/JPG, WEBP, and GIF.
  • Use file_name plus repo_id when needed; Genesis resolves the stored path in the runtime environment.
  • Prompts should describe subject, style, composition, and any constraints such as aspect ratio.
  • Use repo_id when the image lives outside the default uploads repository.

​
Vision and image-generation providers

image_analysis uses the active bot’s vision-capable chat provider when available — for example AWS Bedrock, Snowflake Cortex, OpenAI, Azure OpenAI, or Databricks. Azure OpenAI and Databricks vision calls route through LiteLLM with per-call credentials from the saved providers.azureopenai or providers.databricks connection; Genesis does not build a separate native OpenAI client for image analysis on those providers. image_generation uses the active LiteLLM Proxy when it is the active LLM provider. Genesis selects the first discovered proxy model with the image_generation catalog mode. Genesis AI uses its genesis-ai-image alias; the alias may not appear in the user’s /models response. The proxy request is made with that provider’s credentials (the user’s key on Genesis AI), and an image returned as a URL is downloaded by Genesis. If proxy model discovery fails, generation stops rather than switching providers. If the proxy has no image model, image generation can still use OpenAI or AWS Bedrock credentials from Config → AI Providers, or a dedicated image key via set_image_api_key(). Azure OpenAI is not supported for image generation. See Model Universe for discovered proxy models.

​
Image Analysis API key

When the vision path falls back to a raw OpenAI client (for example OpenAI as the active provider), image_analysis resolves its OpenAI API key in this order:
  1. The caller’s scoped default Image Analysis (OpenAI) connection in Config → Connections (My default, then Role default), using the active conversation’s user and bot for secret access.
  2. Legacy EXT_SERVICE_CONFIG values when no scoped default applies.
When a scoped image_analysis default resolves but secret access fails, Genesis surfaces that error instead of silently using legacy configuration. Save an Image Analysis (OpenAI) connection under Dev Tools, set it as your default or a role default, and run Test Connection before relying on vision analysis in chat. See Connections.

​
Limitations or Notes

  1. image_analysis works on image files only. For PDFs, use the git_action OCR or visual page analysis workflows.
  2. Analysis quality depends on resolution and clarity of the source image.
  3. Very small text or dense tables may require higher resolution inputs.
  4. Generated images may vary from the prompt when instructions are vague.
  5. Large or many images can increase processing time and memory usage.
  6. Corrupted or unsupported files will return detailed error messages.

​
Output

  • Image Analysis Descriptions of objects, text, layout, colors, and relationships. May include extracted text and confidence indicators.
  • Image Generation New image files with artifact ids and links for download or attachment.
  • File Location and Retrieval Resolved paths for found files so downstream tools can display or attach them.
  • Errors Clear messages for missing files, unsupported formats, or processing failures.