The iceberg extension supports attaching Iceberg REST Catalogs. Before attaching an Iceberg REST Catalog, you must install the iceberg extension by following the instructions located in the overview.
If you are attaching to an Iceberg REST Catalog managed by Amazon, please see the instructions for attaching to Amazon S3 Tables or Amazon SageMaker Lakehouse.
For all other Iceberg REST Catalogs, you can follow the instructions below. Please see the Examples section for questions about specific catalogs.
Most Iceberg REST Catalogs authenticate via OAuth2. You can use the existing DuckDB secret workflow to store login credentials for the OAuth2 service.
CREATE SECRET iceberg_secret (
TYPE iceberg,
CLIENT_ID 'admin',
CLIENT_SECRET 'password',
OAUTH2_SERVER_URI 'http://iceberg_rest_catalog_url.com/v1/oauth/tokens'
);
If you already have a Bearer token, you can pass it directly to your CREATE SECRET statement
CREATE SECRET iceberg_secret (
TYPE iceberg,
TOKEN 'bearer_token'
);
You can attach the Iceberg catalog with the following ATTACH statement.
LOAD httpfs;
ATTACH 'warehouse' AS iceberg_catalog (
TYPE iceberg,
SECRET iceberg_secret, -- pass a specific secret name to prevent ambiguity
ENDPOINT 'https://rest_endpoint.com'
);
To see the available tables run
SHOW ALL TABLES;
ATTACH Options
A REST Catalog with OAuth2 authorization can also be attached with just an ATTACH statement. See the complete list of ATTACH options for a REST Catalog below.
| Parameter | Type | Default | Description |
|---|---|---|---|
ENDPOINT_TYPE |
VARCHAR |
NULL |
Used for attaching S3 Tables or Glue catalogs. Allowed values are GLUE and S3_TABLES. |
ENDPOINT |
VARCHAR |
NULL |
URL endpoint to communicate with the REST Catalog. Cannot be used in conjunction with ENDPOINT_TYPE. |
SECRET |
VARCHAR |
NULL |
Name of secret used to communicate with the REST Catalog. |
CLIENT_ID |
VARCHAR |
NULL |
CLIENT_ID used for Secret. |
CLIENT_SECRET |
VARCHAR |
NULL |
CLIENT_SECRET needed for Secret. |
DEFAULT_REGION |
VARCHAR |
NULL |
A Default region to use when communicating with the storage layer. |
OAUTH2_SERVER_URI |
VARCHAR |
NULL |
OAuth2 server url for getting a Bearer Token. |
AUTHORIZATION_TYPE |
VARCHAR |
OAUTH2 |
Pass SigV4 for Catalogs the require SigV4 authorization, none for catalogs that don't need authentication. |
ACCESS_DELEGATION_MODE |
VARCHAR |
vended_credentials |
Access delegation mode. Allowed values are vended_credentials and none. |
EXTRA_HTTP_HEADERS |
MAP |
NULL |
Additional HTTP headers to send with REST Catalog requests. |
SUPPORT_NESTED_NAMESPACES |
BOOLEAN |
true |
Option for catalogs that support nested namespaces. |
SUPPORT_STAGE_CREATE |
BOOLEAN |
false |
Option for catalogs that do not support stage create. |
MAX_TABLE_STALENESS |
INTERVAL |
NULL |
Option for preventing unnecessary requests to the Iceberg REST Catalog. You can pass human readable interval strings. 10 minutes, 30 seconds, 1 year all work. |
PURGE_REQUESTED |
BOOLEAN |
true |
Option to send the PurgeRequested parameter when dropping a table. |
The following options can only be passed to a CREATE SECRET statement and they require AUTHORIZATION_TYPE to be OAUTH2:
| Parameter | Type | Default | Description |
|---|---|---|---|
OAUTH2_GRANT_TYPE |
VARCHAR |
NULL |
Grant Type when requesting an OAuth Token. |
OAUTH2_SCOPE |
VARCHAR |
NULL |
Requested scope for the returned OAuth Access Token. |
Supported Operations
The DuckDB Iceberg extension supports the following operations when used with a REST Catalog attached:
CREATE/DROP SCHEMACREATE/DROP TABLEINSERT INTOUPDATEDELETESELECT
Since these operations are supported, the following will also work:
COPY FROM DATABASE duckdb_db TO iceberg_datalake;
-- Or
COPY FROM DATABASE iceberg_datalake TO duckdb_db;
This functionality enables deep copies between Iceberg and DuckDB storage.
Limitations for UPDATE and DELETE
The UPDATE and DELETE operations have the following limitations:
- They only work on tables that are not partitioned and not sorted. Attempting these operations on partitioned or sorted tables results in an error.
- DuckDB-Iceberg only writes positional deletes. Copy-on-write functionality is not yet supported.
- DuckDB-Iceberg only supports merge-on-read semantics. If a table has
write.update.modeorwrite.delete.modeproperties set to something other thanmerge-on-read, the operation fails.
Metadata Operations
The functions iceberg_metadata and iceberg_snapshots are also available to use with an Iceberg REST Catalog using a fully qualified path, e.g.:
SELECT * FROM iceberg_metadata(my_datalake.default.t)
-- Or
SELECT * FROM iceberg_snapshots(my_datalake.default.t)
This functionality enables the user to do time traveling.
-- Using a snapshot id
SELECT * FROM my_datalake.default.t AT (VERSION => SNAPSHOT_ID)
-- Or using a timestamp
SELECT * FROM my_datalake.default.t AT (TIMESTAMP => TIMESTAMP '2025-09-22 12:32:43.217')
Interoperability with DuckLake
The DuckDB Iceberg extensions exposes a function to do metadata only copies of the Iceberg metadata to DuckLake, which enables users to query Iceberg tables as if they were DuckLake tables.
-- Given that we have an Iceberg catalog attached aliased to iceberg_datalake
ATTACH 'ducklake:my_ducklake.ducklake' AS my_ducklake;
CALL iceberg_to_ducklake('iceberg_datalake', 'my_ducklake');
It is also possible to skip a set of tables provided the skip_tables parameter.
CALL iceberg_to_ducklake('iceberg_datalake', 'my_ducklake', skip_tables := ['table_to_skip']);
Table Properties Functions
DuckDB provides functions to view and modify Iceberg table properties:
| Function | Description |
|---|---|
iceberg_table_properties(table) |
Returns all properties of the specified table. |
set_iceberg_table_properties(table, properties) |
Sets properties on the specified table. |
remove_iceberg_table_properties(table, property_list) |
Removes properties from the specified table. |
-- View table properties
SELECT *
FROM iceberg_table_properties(iceberg_catalog.default.my_table);
-- Set table properties
CALL set_iceberg_table_properties(
iceberg_catalog.default.my_table,
{'write.update.mode': 'merge-on-read', 'write.delete.mode': 'merge-on-read'}
);
-- Remove table properties
CALL remove_iceberg_table_properties(
iceberg_catalog.default.my_table,
['some.property']
);
You can also create a table with table properties.
CREATE TABLE test_create_table (a INTEGER)
WITH (
'format-version' = '2', -- format version will be elevated to format-version when creating a table
'location' = 's3://path/to/data', -- location will be elevated to location when creating a table
'property1' = 'value1',
'property2' = 'value2'
);
Unsupported Operations
The following operations are not supported by the DuckDB Iceberg extension:
MERGE INTOALTER TABLE
Specific Catalog Examples
Cloudflare R2 Catalog
To attach to an R2 Cloudflare managed catalog follow the attach steps below.
CREATE SECRET r2_secret (
TYPE iceberg,
TOKEN 'r2_token'
);
You can create a token by following the create an API token steps in getting started. Then, attach the catalog with the following commands.
ATTACH 'warehouse' AS my_r2_catalog (
TYPE iceberg,
ENDPOINT 'catalog-uri'
);
The variables for warehouse and catalog-uri are available under the settings of the R2 Object Storage Catalog (R2 Object Store, Catalog name, Settings).
Once you attached to the R2 Data Catalog, create a schema. You can set it as default with the USE command:
CREATE SCHEMA my_r2_catalog.my_schema;
USE my_r2_catalog.my_schema;
Polaris
To attach to a Polaris catalog, use the following commands:
CREATE SECRET polaris_secret (
TYPE iceberg,
CLIENT_ID 'admin',
CLIENT_SECRET 'password',
);
ATTACH 'quickstart_catalog' AS polaris_catalog (
TYPE iceberg,
ENDPOINT 'polaris_rest_catalog_endpoint',
ACCESS_DELEGATION_MODE 'vended_credentials'
);
Lakekeeper
To attach to a Lakekeeper catalog the following commands will work.
CREATE SECRET lakekeeper_secret (
TYPE iceberg,
CLIENT_ID 'admin',
CLIENT_SECRET 'password',
OAUTH2_SCOPE 'scope',
OAUTH2_SERVER_URI 'lakekeeper_oauth_url'
);
ATTACH 'warehouse' AS lakekeeper_catalog (
TYPE iceberg,
ENDPOINT 'lakekeeper_irc_url',
SECRET 'lakekeeper_secret'
);
Google Cloud BigLake
To attach to a Google Cloud BigLake catalog, you can use extra HTTP headers to specify the GCP project for billing purposes.
First, get your Google Cloud access token:
gcloud auth application-default print-access-token
Then create a secret with the token and extra headers:
CREATE SECRET biglake_secret (
TYPE iceberg,
TOKEN 'your_access_token',
EXTRA_HTTP_HEADERS MAP {
'x-goog-user-project': 'your_gcp_project_id'
}
);
Attach to the BigLake catalog:
ATTACH 'gs://your-biglake-bucket' AS biglake_catalog (
TYPE iceberg,
ENDPOINT 'https://biglake.googleapis.com/iceberg/v1/restcatalog',
SECRET biglake_secret
);
Example using the BigLake public dataset:
CREATE SECRET biglake_public_secret (
TYPE iceberg,
TOKEN 'your_access_token',
EXTRA_HTTP_HEADERS MAP {
'x-goog-user-project': 'your_gcp_project_id'
}
);
ATTACH 'gs://biglake-public-nyc-taxi-iceberg' AS biglake_public (
TYPE iceberg,
ENDPOINT 'https://biglake.googleapis.com/iceberg/v1/restcatalog',
SECRET biglake_public_secret
);
-- Query the data
SELECT count(*) FROM biglake_public.public_data.nyc_taxicab;
Note: Google Cloud access tokens expire after 1 hour. For long-running sessions, you'll need to refresh the token periodically.
Limitations
DuckDB supports Iceberg REST Catalogs backed by S3, S3 Tables, and Google Cloud Storage (GCS). Support for other storage backends is not yet available.