Manage Document Indexes over REST

Overview

Genesis stores searchable document content in document indexes backed by ChromaDB. The same indexes power the Document Indexes page in the dashboard, the agents’ “search the docs” abilities, the “Ask” feature, and the Knowledge Base feature that bundles indexes for assignment to specific agents. This page covers the REST surface for indexes. Two equivalent paths are available — pick whichever fits your client:

REST endpoints (this guide) — typed schemas, REST verbs.
All-in-one tool function via /api/tool_call with {"func": "_document_index", "func_args": {"action": "...", ...}}. Same code path the bots use.

For background on what indexes are and how they fit with knowledge bases, see the user-guide Document Indexes page. For the field-by-field schema of the destructive DELETE flow, see the API reference Document index management endpoints section.

Authenticate with a Personal Access Token. See Personal Access Tokens.Index and document mutations require the manage_document_indexes permission. Other read-only listing, search, and ask endpoints are available to any authenticated caller; polling background upload status also requires manage_document_indexes.

export GENESIS_URL="https://<your-genesis-host>"
export GENESIS_PAT="gp_replace-me"
AUTH=(-H "Authorization: Bearer $GENESIS_PAT")
JSON=(-H "Content-Type: application/json")

Endpoint summary

Operation	Method & Path	Permission
List indexes	`GET /api/indexes`	any auth
Create index	`POST /api/indexes`	`manage_document_indexes`
Rename index	`PUT /api/indexes/{index_name}/rename`	`manage_document_indexes`
Delete index (async)	`DELETE /api/indexes/{index_name}`	`manage_document_indexes`
Inspect delete impact	`GET /api/indexes/{index_name}/delete-impact`	`manage_document_indexes`
Poll async delete status	`GET /api/indexes/operations/{operation_id}`	`manage_document_indexes`
List documents in an index	`GET /api/indexes/{index_name}/documents`	any auth
Add document to index (async)	`POST /api/indexes/{index_name}/documents`	`manage_document_indexes`
Poll background upload status	`GET /api/indexes/{index_name}/uploads`	`manage_document_indexes`
Get docs navigation tree	`GET /api/indexes/{index_name}/navigation`	any auth
Get document content	`GET /api/indexes/{index_name}/documents/{file_id}/content`	any auth
Rename a document	`PUT /api/indexes/{index_name}/documents/{file_id}/rename`	`manage_document_indexes`
Delete a document	`DELETE /api/indexes/{index_name}/documents/{file_id}`	`manage_document_indexes`
Search (semantic)	`POST /api/search`	any auth
Ask (RAG, LLM-generated answer)	`POST /api/ask`	any auth

{file_id} in path templates accepts URL-encoded values, including slashes inside the id. The destructive DELETE flow (DELETE /api/indexes/{index_name}, /delete-impact, and /operations/{operation_id}) is documented in detail in the API Reference — document index management endpoints.

Indexes

List indexes

curl -sS "${AUTH[@]}" "$GENESIS_URL/api/indexes"

{
  "success": true,
  "data": [
    {
      "name": "wiki",
      "file_count": 142,
      "created_at": "2026-03-08T12:30:00Z",
      "needs_reindex": false
    },
    {
      "name": "legacy_kb",
      "file_count": 58,
      "created_at": "2026-02-01T10:00:00Z",
      "needs_reindex": true
    }
  ],
  "state": null
}

When no document manager can be built because embeddings are not configured, data is an empty array and state carries the embedding identity discriminator (for example "unset" or "incomplete"). The dashboard uses this to show the no-embedding banner instead of treating the inventory as genuinely empty. Omit state (or receive null) on successful reads when a manager is active. Each item in data includes:

Field	Type	Default	Description
`name`	string	—	Index name.
`file_count`	integer	—	Number of indexed files.
`created_at`	string	—	ISO timestamp for when the index was created (or earliest indexed document when creation metadata is missing).
`needs_reindex`	boolean	`false`	`true` when the index exists only in a prior embedding-model Chroma collection and is not yet searchable with the active embedding model.

Document embeddings are stored in model-specific collections named {provider}_{sanitized_model}_index. When the active embedding model changes, listing and search use the new collection. Indexes stranded in a prior collection still appear here with needs_reindex: true instead of vanishing from inventory. Saving a new embedding configuration on Config → AI Providers → Embeddings starts a full background re-embed of those documents into the active collection. See Setup AI Model — Document indexes and embedding-model changes.

On Databricks Apps with durable document indexes configured, a missing index may be restoring from its volume snapshot after startup. Search then returns no results with a restore-in-progress message; wait and retry. Mutations to an index awaiting restore are refused until it is available. See Durable document indexes on Databricks.

Create an index

curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{"index_name": "customer_support_docs"}' \
  "$GENESIS_URL/api/indexes"

{ "success": true, "message": "Index 'customer_support_docs' created successfully" }

Index creation requires a resolvable embedding identity. When none is configured, the route returns HTTP 409 with error_type: "embedding_not_configured". When a pointer is set but the provider lacks credentials, the response uses error_type: "provider_not_configured". See Setup AI Model and Embedding provider validation and coherence. On Databricks Apps, creation and rename can also return HTTP 503 while snapshot restoration is pending or when the volume cannot remove a saved copy; retry after restoration or when volume access is available.

Rename an index

curl -sS -X PUT "${AUTH[@]}" "${JSON[@]}" \
  -d '{"new_index_name": "customer_support_docs_v2"}' \
  "$GENESIS_URL/api/indexes/customer_support_docs/rename"

Successful create, rename, and completed delete operations refresh Genesis capability discovery immediately. After a mutation, agents can find the updated index list through manage_context(action="SEARCH_CONTEXT", search_scope="document_indexes") without a service restart.

Delete an index

Index deletion is destructive and runs in the background. The DELETE returns HTTP 202 with an operation_id you poll. Use GET /api/indexes/{index_name}/delete-impact first to see which knowledge bases and agents would be affected. Genesis removes the index from every embedding-model Chroma collection that holds it — the active collection and any prior model-specific collections where stranded chunks still exist (the same indexes listed with needs_reindex: true). Stranded copies are purged first; attached files are detached only after every physical copy is confirmed gone. Progress polling may show steps such as Deleting from prior embedding collection (1/2). The full schema for the delete flow lives in the API reference: Document index management endpoints. A typical end-to-end delete:

# 1. See what would be affected.
curl -sS "${AUTH[@]}" "$GENESIS_URL/api/indexes/wiki/delete-impact"

# 2. Start the delete.
DELETE_OP=$(curl -sS -X DELETE "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/wiki" | jq -r .operation_id)

# 3. Poll until it finishes.
while :; do
  R=$(curl -sS "${AUTH[@]}" \
    "$GENESIS_URL/api/indexes/operations/$DELETE_OP")
  echo "$R" | jq '{status, progress, current_step}'
  STATUS=$(echo "$R" | jq -r .status)
  case "$STATUS" in
    completed) break ;;
    failed)    echo "FAILED: $(echo "$R" | jq -r .error_message)" >&2; exit 1 ;;
  esac
  sleep 2
done

Documents

List documents in an index

curl -sS "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/wiki/documents"

# Optional ?query= filters by name/path.
curl -sS "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/wiki/documents?query=runbook"

# Page through large indexes (50 documents per page maximum).
curl -sS "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/wiki/documents?start=0&count=50"

Query params:

Param	Default	Notes
`query`	none	Optional filter by name or path substring.
`start`	`0`	Zero-based document offset for this page.
`count`	`50`	Requested page size. Responses are limited to 50 documents per page.

The response includes pagination metadata plus a data array of document items. Each item includes file_id, path, relative_path, file_name, file_type, file_size, mimetype, is_binary, and created_at. Example response (truncated):

{
  "success": true,
  "data": [ { "file_id": "...", "path": "wiki/ops/runbook.md", "file_name": "runbook.md" } ],
  "total_count": 142,
  "start": 0,
  "count": 50,
  "actual_count": 50,
  "has_more": true,
  "next_start": 50,
  "message": "Showing first 50 of 142 items. 92 more items exist.",
  "state": null
}

When no document manager is active for the requested index, data is empty, pagination fields reflect the empty page, and state carries the embedding identity discriminator (same semantics as GET /api/indexes). When has_more is true, request the next page with start=<next_start> until has_more is false. The dashboard Config → Documents file list uses the same endpoint and follows every page automatically.

Add a document

You can add a document either by referencing a file already known to the file manager (file_id) or by giving a direct filepath. Provide exactly one of the two.

ADD_DOCUMENTS uses the document-index exclusion rules described in the Document Index Tools guide. An explicitly named excluded path returns HTTP 400; a filename that merely contains words such as target, build, or .o is accepted unless it matches an excluded path segment or filename pattern.

# By file_id (recommended; upload first, then add).
curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{
        "file_id": "123e4567-e89b-12d3-a456-426614174000",
        "conflict_policy": "overwrite_on_conflict",
        "include_binary": true
      }' \
  "$GENESIS_URL/api/indexes/wiki/documents"

# By direct filepath on the server.
curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{
        "filepath": "/srv/genesis/manual/runbook.md",
        "conflict_policy": "error_on_conflict"
      }' \
  "$GENESIS_URL/api/indexes/wiki/documents"

Conflict policies (DocumentConflictPolicy):

Value	Behaviour
`error_on_conflict` (default)	Reject the add if a document with the same name already exists.
`overwrite_on_conflict`	Replace the existing document.

The route returns HTTP 202 Accepted after checking the request and queueing the upload; it does not wait for embedding or indexing to finish. Uploads are written one at a time in arrival order across indexes in the process. The document becomes available in the indexed document list and search after indexing completes. Use the returned operation_id to locate the upload in GET /api/indexes/{index_name}/uploads results:

{
  "success": true,
  "data": {
    "message": "Document 'runbook.md' queued for indexing",
    "file_id": "123e4567-e89b-12d3-a456-426614174000",
    "created_at": "2026-04-21T18:30:00Z",
    "operation_id": "7da7ab38-81b4-459e-b37b-47a82691cb85",
    "status": "queued"
  }
}

For a direct filepath add, file_id can be null until indexing finishes. The upload-status response lists records oldest first:

Field	Type	Description
`operation_id`	string	Background upload identifier.
`index_name`	string	Target index.
`file_id`	string or `null`	File-manager ID, which may be `null` until a filepath add finishes.
`file_name`	string	Document name.
`status`	string	`queued`, `indexing`, `ready`, or `failed`.
`queued_at`	string	Acceptance timestamp.
`started_at`	string or `null`	Indexing start timestamp.
`completed_at`	string or `null`	Completion timestamp.
`message`	string or `null`	Result message when status is `ready`.
`error_message`	string or `null`	Failure reason when status is `failed`.
`error_type`	string or `null`	API error type when status is `failed`.

The service keeps active uploads and up to 200 recently finished uploads in process memory. A server restart loses queued work and its status records; a file accepted but not yet indexed remains in the file store and must be submitted again.

curl -sS -H "Authorization: Bearer $GENESIS_PAT" \
  "$GENESIS_URL/api/indexes/wiki/uploads"

Example response:

{
  "success": true,
  "data": [
    {
      "operation_id": "7da7ab38-81b4-459e-b37b-47a82691cb85",
      "index_name": "wiki",
      "file_id": "123e4567-e89b-12d3-a456-426614174000",
      "file_name": "runbook.md",
      "status": "indexing",
      "queued_at": "2026-04-21T18:30:00Z",
      "started_at": "2026-04-21T18:30:01Z",
      "completed_at": null,
      "message": null,
      "error_message": null,
      "error_type": null
    }
  ]
}

The dashboard’s Config → Documents upload drawer polls while uploads are queued or indexing. See the Document Indexes guide for the dashboard experience. If you need to add many files at once, use glob patterns / recursive walks / Git-repo sources via the all-in-one tool form:

curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{
        "func": "_document_index",
        "func_args": {
          "action": "ADD_DOCUMENTS",
          "index_name": "code",
          "filepath": "src/**/*.py",
          "recursive": true,
          "conflict_policy": "overwrite_on_conflict"
        }
      }' \
  "$GENESIS_URL/api/tool_call"

Rename a document

FILE_ID="123e4567-e89b-12d3-a456-426614174000"
curl -sS -X PUT "${AUTH[@]}" "${JSON[@]}" \
  -d '{"new_filename": "runbook-v2.md"}' \
  "$GENESIS_URL/api/indexes/wiki/documents/$FILE_ID/rename"

Delete a document

curl -sS -X DELETE "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/wiki/documents/$FILE_ID"

Get document content (with pagination)

# Default: ChromaDB chunks (see caveat below).
curl -sS "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/wiki/documents/$FILE_ID/content?count=5"

# Read by line range.
curl -sS "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/wiki/documents/$FILE_ID/content?pagination_mode=lines&start=0&count=200"

# Read by character range.
curl -sS "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/wiki/documents/$FILE_ID/content?pagination_mode=chars&start=0&count=20000"

Query params:

Param	Default	Notes
`pagination_mode`	`null`	`lines` (code/structured), `chars` (plain text), or empty for ChromaDB chunks.
`start`	`0`	Starting position (line / char / chunk).
`count`	mode-specific (100 lines / 10 000 chars / 20 chunks)	How much to read.

With no pagination_mode, the response is a window over ChromaDB chunks. Chunks may be reordered or de-duplicated relative to the on-disk file. For exact, ordered file content prefer pagination_mode=lines, or fetch the original file via the /api/files/{file_id} endpoint.

For UI consumers that render a docs site:

curl -sS "${AUTH[@]}" \
  "$GENESIS_URL/api/indexes/docs/navigation"

Returns an ordered list of groups, each containing pages with stable ids and titles, plus an optional home_page_id.

Search and Ask

Semantic search

curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{"query": "How do I rotate the LLM proxy key?"}' \
  "$GENESIS_URL/api/search?index_name=wiki&top_n=5&optimize_results_for=top_chunks"

Query params:

Param	Default	Notes
`index_name`	none	Restrict search to one index. Omit to search every index.
`top_n`	`10`	1–50.
`optimize_results_for`	`top_chunks`	`top_chunks` (default; may include several chunks per doc) or `top_docs` (top chunk per unique document — more diverse results).

Response:

{
  "success": true,
  "data": [
    {
      "score": 0.91,
      "text":  "Rotating the LiteLLM proxy key…",
      "metadata": { "source": "wiki/ops/litellm.md", "file_id": "..." }
    }
  ]
}

Ask (RAG with LLM-generated answer)

curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{"query": "What is our deployment process for SPCS?"}' \
  "$GENESIS_URL/api/ask?top_n=3"

top_n is 1–10 (default 1) and controls how many top documents are used to ground the LLM answer. Response:

{
  "success": true,
  "data": {
    "answer": "We deploy via …",
    "references": "wiki/ops/spcs.md, wiki/release/release.md"
  }
}

/api/ask requires an LLM backend to be configured. On a fresh install without an LLM provider this call returns an error.

Knowledge Bases (assigning indexes to agents)

A Knowledge Base is a named bundle of one or more document indexes that you can assign to specific agents. Use the Knowledge Base Admin Tools or the /api/knowledge-bases REST routes after creating the indexes you want to expose:

curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{"title": "Support KB", "index_names": ["wiki", "support_docs"]}' \
  "$GENESIS_URL/api/knowledge-bases"

Then assign the KB to an agent through the agent settings endpoints (/api/bots/{bot_id}/...). The dashboard’s Knowledge Bases flow does the same thing interactively.

Git-backed document indexes

When a managed source registers documents in scan_types_enabled, Genesis syncs markdown from documents_root into document indexes and routes UI and agent edits through the managed workspace (/api/managed-sources/{source_id}/workspace/*). Use POST /api/managed-sources/preview-documents before first sync to see how repo folders map to target indexes, and pass overwrite_indexes on sync when adoption would replace index-only documents that are not yet in Git. For long-running syncs, use POST /api/managed-sources/{source_id}/sync-async and poll GET /api/managed-sources/{source_id}/sync-status/{job_id} instead of the synchronous POST .../sync route. For agent tooling, use propose_kb_edit and open_kb_pull_request via /api/tool_call — see Document Index Tools. Full REST field reference: API Reference — unified managed sources.

Working through `/api/tool_call`

If you prefer parity with what the bots see, every action above is available via the single _document_index tool. The action names are upper-snake-case:

ADD_DOCUMENTS, ADD_TEXT, CREATE_INDEX, LIST_INDICES, DELETE_INDEX,
LIST_DOCUMENTS, SEARCH, RENAME_INDEX, ASK, DELETE_DOCUMENT,
DELETE_BY_METADATA, GET_DOCUMENT, RENAME_DOCUMENT, GET_INDICES_STATS

Examples:

# LIST_INDICES
curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{"func": "_document_index", "func_args": {"action": "LIST_INDICES"}}' \
  "$GENESIS_URL/api/tool_call"

# GET_DOCUMENT, line-mode
curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{
        "func": "_document_index",
        "func_args": {
          "action": "GET_DOCUMENT",
          "index_name": "wiki",
          "file_id": "123e…",
          "pagination_mode": "lines",
          "start": 0,
          "count": 200
        }
      }' \
  "$GENESIS_URL/api/tool_call"

# DELETE_BY_METADATA — bulk delete every entry from one Snowflake table
curl -sS -X POST "${AUTH[@]}" "${JSON[@]}" \
  -d '{
        "func": "_document_index",
        "func_args": {
          "action": "DELETE_BY_METADATA",
          "index_name": "snowflake_metadata",
          "metadata": {"qualified_name": "snowflake://DB/SCHEMA/TABLE"}
        }
      }' \
  "$GENESIS_URL/api/tool_call"

The full action list with parameter docs is in Document Index Tools.

Common pitfalls

409 Conflict with embedding_not_configured or provider_not_configured on a write call. No embedding identity resolves, or the active provider is not fully configured. Configure embeddings on Config → AI Providers → Embeddings, recover a stranded corpus, or set embedding.active_provider / embedding.active_model in genesis-config.yaml. Genesis does not infer an OpenAI embedding fallback from chat configuration.
GET /api/indexes returns "data": [] with a non-null state. The list is empty because embeddings are not configured, not because no indexes exist. Check state (unset or incomplete) before assuming a greenfield deployment.
403 Forbidden on a write call. The PAT’s user is missing the manage_document_indexes permission. Add the matching role in Users and Roles and try again.
Only the first page of documents is visible. GET /api/indexes/{index_name}/documents and _document_index action LIST_DOCUMENTS return at most 50 documents per request. Use start and count (or follow next_start until has_more is false) to reach every file in large indexes.
Documents appear in LIST_DOCUMENTS but SEARCH finds nothing. Embeddings must be configured. See Setup AI Model. After a restart, search server logs or Pulse Live Logs for failed startup smoke test — Genesis runs a one-time embedding connection check at boot and logs misconfigured providers before the first index call.
An index is listed with needs_reindex: true or shows a Needs reindex badge in the dashboard. The index still exists in a prior embedding-model collection. Wait for background re-embedding after an embedding change, or switch back to the prior embedding model to restore the original collection without re-embedding. See Setup AI Model — Document indexes and embedding-model changes.
ASK returns an error. The deployment has no LLM provider configured for ask-style answers. Set one up in Setup AI Model.
GET_DOCUMENT returns garbled or out-of-order content. That is the ChromaDB-chunk caveat — switch to pagination_mode=lines or fetch the original file via /api/files/{file_id}.
Deleting an index appears stuck at low progress. Large indexes can take a while; poll /api/indexes/operations/{operation_id}. Progress may include Deleting from prior embedding collection when the index still exists in a prior embedding-model collection. If status becomes failed, inspect error_message — it may list collection names that still hold the index (for example Index 'wiki' still exists in openai_text_embedding_3_large_index; retry to finish removing it). Issue another DELETE /api/indexes/{index_name} to finish removal after fixing storage access.

Getting Started

Getting to Know Genesis

Creating A Mission

User Guide

Blueprint

Genesis Data Agents

Genesis Prover

Genesis Apps

Genesis REST API

Genesis Agent Security Controls

Genesis Data Agent's Toolkit

Setup

Slack and Teams

Data Connections

Deployment Options

Manage Document Indexes over REST

Overview

Endpoint summary

Indexes

List indexes

Create an index

Rename an index

Delete an index

Documents

List documents in an index

Add a document

Rename a document

Delete a document

Navigation tree

Search and Ask

Semantic search

Ask (RAG with LLM-generated answer)

Knowledge Bases (assigning indexes to agents)

Git-backed document indexes

Working through `/api/tool_call`

Common pitfalls

​Overview

​Endpoint summary

​Indexes

​List indexes

​Create an index

​Rename an index

​Delete an index

​Documents

​List documents in an index

​Add a document

​Rename a document

​Delete a document

​Get document content (with pagination)

​Navigation tree

​Search and Ask

​Semantic search

​Ask (RAG with LLM-generated answer)

​Knowledge Bases (assigning indexes to agents)

​Git-backed document indexes

​Working through /api/tool_call

​Common pitfalls

​Related pages

Overview

Endpoint summary

Indexes

List indexes

Create an index

Rename an index

Delete an index

Documents

List documents in an index

Add a document

Rename a document

Delete a document

Get document content (with pagination)

Navigation tree

Search and Ask

Semantic search

Ask (RAG with LLM-generated answer)

Knowledge Bases (assigning indexes to agents)

Git-backed document indexes

Working through `/api/tool_call`

Common pitfalls

Related pages