The unity_catalog extension adds support for the Unity Catalog atop the Delta Lake format and DuckDB Delta extension.

The delta extension adds support for the Delta Lake open-source storage format. It is built using the Delta Kernel. The extension offers read support for Delta tables, both local and remote.

For implementation details, see the announcement blog post.

Warning Both the unity_catalog and delta extensions are currently experimental and only supported on given platforms.

Installing and Loading

To install and load, run:

INSTALL unity_catalog;
LOAD unity_catalog;

Usage

Given that you already have a Unity Catalog setup with either Databricks or Unity Catalog OSS, you will need to configure your secret token, endpoint, and region; then attach to your catalog. For example an AWS configuration would look like this:

CREATE SECRET uc (
    TYPE unity_catalog,
    TOKEN 'token',
    ENDPOINT 'endpoint',
    AWS_REGION 'region'
);
ATTACH 'test_catalog' AS test_catalog (TYPE unity_catalog, DEFAULT_SCHEMA 'main');

Where token comes from your Databricks or OSS Unity Catalog deployment, and endpoint is your Unity Catalog REST API endpoint.

For more details on these deployments see Databricks Unity Catalog Docs and OSS Unity Catalog Docs.

To confirm correct attachment, try something like:

SHOW ALL TABLES;
SELECT * FROM test_catalog.test_schema.test_table LIMIT 10;

Features

This extension is still experiment and work-in-progress; it supports:

• Listing available tables (SHOW ALL TABLES;)
• Interacting with tables using standard SQL (SELECT * FROM <catalog>.<schema>.<table>;)
• Time travel (SELECT * FROM .. AT (VERSION => ..);)
• Inserts (INSERT INTO .. VALUES (..);)

It does not currently support:

• DELETE or UPDATE
• Creation or manipulation of TABLEs VIEWs or SCHEMAs

Supported DuckDB Versions and Platforms

The unity_catalog (and delta) extension currently supports the following platforms:

• Linux AMD64 (x86_64 and ARM64): linux_amd64 and linux_arm64
• macOS Intel and Apple Silicon: osx_amd64 and osx_arm64
• Windows AMD64: windows_amd64

Support for the other DuckDB platforms is work-in-progress.