​
Overview

​
Tool Name

image_tools

​
Purpose

The image_analysis_and_generation_tools offer functionalities to analyze, interpret, and generate images. These tools enable extracting insights from visual data and creating new, customized images for use in various workflows such as reporting, marketing, visual storytelling, and automation.

​
Functions Available

  1. _image_analysis: Analyzes an image for its content and provides answers to queries about the image.

  2. image_generation: Generates an image based on a descriptive prompt.

​
Key Features

Image Analysis

Analyze images to extract embedded text, objects, scenes, or other features.

Query-Based Insights

Respond to specific queries about the contents of an image.

Custom Image Generation

Generate custom images from textual prompts, supporting creative visual outputs.

Multi-Format Support

Enable multi-format image handling, including base64 or local file paths, for analysis and generation.

​
Input Parameters

_image_analysisAnalyze an Image for Content & Respond to Queries
Input ParametersDefinitionFormat
query (Optional)A question or query about the image (e.g., “What does this image contain?”).String
openai_file_id (Optional)File ID of the image if already uploaded to OpenAI.String
file_name (Optional)Local file path of the image to be analyzed.String
image_generationCreate a New Image Based on a Descriptive Prompt
Input ParametersDefinitionFormat
promptA detailed description of the image to generate (e.g., “A futuristic cityscape with neon lights”).String

​
Output

  • Image Analysis

    • Provides answers to queries about the image, including detected objects, text, and relevant metadata (dimensions, formats).

  • Image Generation

    • Produces a high-quality visual representation of the prompt.

    • Returns either a direct file link, a base64-encoded output, or a success confirmation with details for further use.

Genbot Tip

  • For text-based queries, consider using OCR capabilities in _image_analysis to extract textual content from images (e.g., scanned documents, receipts).

  • Provide clear, specific prompts in image_generation to achieve the most accurate or creative outputs.

​
How It Works

For analysis, users submit an image file path or use the OpenAI file ID to analyze image content based on a specific query. The tool processes the visual data and responds with results—such as objects detected, text content, or basic metadata. For generation, users provide a descriptive prompt, and the tool returns a newly created image reflecting the specified scene or concept.

IMPORTANT NOTE

  • The quality and relevance of generated images depend on the clarity and specificity of the textual prompt.

  • Analysis of very complex images may yield partial insights or generic results.

  • Large or high-resolution images may slow processing; consider optimizing image sizes before analysis.

  • Generated images may not always adhere to real-world constraints, as they are creatively produced by the model.

_image_analysis – Offers tools to interpret visual images and pictures.

​
Example on Streamlit

In this example, we’ll ask the Genbot to analyze an Entity-Relationship (ER) Diagram for a Hospital Management System.

Upon image analysis and understanding the content within, you can further ask the Genbot how you can use the data from the image to enrich the data in your database/ projects.

​
Example on GenesisAPI

User: Tell me what you see in this image

Genbot:🧰 Using tool: AnalyzeImage...

The image illustrates an Entity-Relationship (ER) Diagram for a "Hospital Management System," highlighting relationships and key attributes between three primary entities: Doctor, Patient, and Medicine. Here's a breakdown:

1. Entities and Their Attributes:
Doctor:
Attributes:
Doctor_id
Name (composite attribute with parts: F.N (First Name), M.N (Middle Name), and L.N (Last Name))
Address
Phone_No (multi-valued attribute)
Relationships:
Treats → Patient
Patient:
Attributes:
Patient_id
Name (composite attribute: F.N, M.N, L.N)
Phone_No (multi-valued attribute)
Address
Bill
Relationships:
Treated by → Doctor
Prescribed → Medicine
Medicine:
Attributes:
Quantity
Price
2. Relationship Details:
Arrows are used to depict the interactions between these entities.
Some attributes are labeled "multi-value" (e.g., Phone_No) or "composite" (e.g., Name components).
This diagram illustrates how Doctors, Patients, and Medicines interconnect within the system, aiding in the data organization and workflow of a hospital management system.

Let me know if you'd like to explore deeper insights or actions based on this diagram! 🏥✨

​

​
Example on Slack

Call your Genbot with @mention and ask to use analyze_image on your image of choice.

In this example, we’ll be calling on Stuart:

You can utilize the data your Genbot receives from analyzing an image to run a deeper analysis, especially if you have an existing data table that can use the information within the image.

​
Generate Image

​
Overview

image_generation – Create images from textual descriptions using DALL-E 3.

​
How To Use

Call your Genbot with @mention and ask to generate an image of your choice.

​
Example on Streamlit

​
Example on GenesisAPI

User: Can you generate an image of a bot waving?

Genbot: 🧰 Using tool: GenerateImage...

​
Example on Slack

In this example, we’ll be making a headshot for Maaaarty:

Harvest Control ToolsJira Connector Tools