S3 Iceberg Import

To load an Iceberg file from S3, both the httpfs and iceberg extensions are required. They can be installed use the INSTALL SQL command. The extensions only need to be installed once.

INSTALL httpfs;
INSTALL iceberg;

To load the extensions for usage, use the LOAD command:

LOAD httpfs;
LOAD iceberg;

Credentials

After loading the extensions, set up the credentials and S3 region to read data. You may either use an access key and secret, or a token.

CREATE SECRET (
    TYPE S3,
    KEY_ID 'AKIAIOSFODNN7EXAMPLE',
    SECRET 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
    REGION 'us-east-1'
);

Alternatively, use the aws extension to retrieve the credentials automatically:

CREATE SECRET (
    TYPE S3,
    PROVIDER CREDENTIAL_CHAIN
);

Loading Iceberg Tables from S3

After the extensions are set up and the S3 credentials are correctly configured, Iceberg table can be read from S3 using the following command:

SELECT *
FROM iceberg_scan('s3://⟨bucket⟩/⟨iceberg-table-folder⟩/metadata/⟨id⟩.metadata.json');

Note that you need to link directly to the manifest file. Otherwise you’ll get an error like this:

Error: IO Error: Cannot open file "s3://⟨bucket⟩/⟨iceberg-table-folder⟩/metadata/version-hint.text": No such file or directory

About this page

Last modified: 2024-05-03

Prerequisites

Credentials

Loading Iceberg Tables from S3

About this page

In this article