Downloads 1.8kthis week

GitHub stars 2

Extension descriptor (YAML)

Read and write Databricks tables directly from DuckDB.

Compatibility Matrix

This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.

Installing and Loading

INSTALL databricks FROM community;
LOAD databricks;

Example

INSTALL databricks FROM community;
LOAD databricks;

CREATE SECRET dbx (
    TYPE databricks,
    HOST 'adb-123.azuredatabricks.net',
    WAREHOUSE_ID 'abc123',
    TOKEN 'dapi...'
);
ATTACH 'main' AS db (TYPE databricks, SECRET dbx, SCHEMA 'sales');

SELECT * FROM db.orders;

INSERT INTO db.orders
SELECT * FROM read_parquet('orders/*.parquet');

About databricks

The Databricks extension attaches one Unity Catalog catalog as a DuckDB database. Tables are read and written with ordinary DuckDB SQL through the SQL Statement Execution API. It does not read Delta files, and it does not open a Thrift session. Filters, projections, LIMIT, and ORDER BY ... LIMIT can be pushed down to the warehouse.

Features include:

• ATTACH support for a Databricks catalog, including secrets for a personal access token or a service principal
• reads from attached schemas and tables
• writes with INSERT, COPY, CREATE TABLE AS, UPDATE, DELETE, and TRUNCATE
• databricks_query and databricks_execute for Databricks SQL that DuckDB cannot express

Added Functions

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings