Skip to main content

​
Overview

Connect Genesis to your Databricks SQL Warehouse with token-based authentication. You’ll collect three values from Databricks, ask a genbot (e.g., Eve) to add the connection, verify it, and then optionally configure harvesting so Data Engineering, DataOps, and Business Analysts can search and explore metadata.

​
Prerequisites

  • A Databricks workspace with access to a SQL Warehouse
  • Permission to create Personal Access Tokens (PATs)
  • Your Genesis environment running with the Data Connector Tools enabled

​
Gather your connection values

You need the following:
SettingWhere to find itExample
Server hostnameDatabricks → SQL Warehouse → Connection detailsdbc-123abc-xyz.cloud.databricks.com
HTTP pathDatabricks → SQL Warehouse → Connection details/sql/1.0/warehouses/abcd1234efgh5678
Access token (PAT)Databricks → User Settings → Developer → Access tokensdap....

​
Create a Personal Access Token (PAT)

1

Open User Settings

In Databricks, click your avatar → User SettingsDb 1 PnSelect Developer underUserDb 2 Pn
2

Generate token

Go to Access tokens → ManageDb 3 PnEnter a comment (description) for your token and set an expiration periodClick GenerateDb 4 Pn
3

Copy token

Copy the token now (you won’t be able to view it again later.)

​
Ask Eve to connect to Databricks using a personal access token:

Eve, add a connection to Databricks called my_databricks, 
my server hostname is MY_SERVER_HOSTNAME, my HTTP path is MY_HTTP_PATH, 
and my access token is MY_ACCESS_TOKEN. Allow all bots to have access.
Example: 
Eve, add a connection to Databricks called my_databricks, 
my server hostname is dbc-209b1505-de07.cloud.databricks.com, 
my HTTP path is /sql/1.0/warehouses/ffb2c2527f699e61, 
and my access token is abc123123123123123123123abc. Allow all bots to have access.
Eve will indicate the connection was successful, or any issues she encounters.
It’s ok if it takes her a few attempts to make the connection string correctly.
Db 5 Pn You can then check the Database Connections screen in Genesis GUI to see the new connection. If you have any trouble connecting, you can ask Eve to provide diagnostic commands you can use to validate your login, authentication, endpoints, etc.

​
Verify the connection

After setting up your connection, you can ask Eve to list database connections. 
Confirm that it has been properly added. Example: 
Eve, list database connections.
Confirm my_databricks is present. Db 6 Pn

​
Check that Genesis can see your data

Example: 
Eve, list the tables and schemas in my_databricks
Db 7 Pn The data will not yet be harvested, so Search Metadata or Data Explorer will not find it.
But it should be able to query directly to find it.

If you want to add it to the harvest, you should do that next. 

​
Add Databricks Catalogs and Schemas to the Harvest

Harvesting makes objects discoverable for search/exploration.
  • After establishing the connection, you can add any catalogs and schemas you’d like to the harvest.
  • Recommend excluding system catalogs unless you’re intentionally analyzing detailed schema and usage information.
Example: 
Eve, add the workspace catalog's healthcare_claims schema to the harvest.
Db 8 Pn

​
Check if the data has been harvested

After waiting a couple of minutes, ask: 
Check to see if this data has been harvested
Db 9 Pn

​
Now see if search metadata can find it:

Example: 
Search Databricks for data on patients
Db 10 Pn

​
Make sure you can query it:

Example: 
How many patients are in the table?
Db 11 Pn