⌘+k ctrl+k
1.2.0 (stable)
Search Shortcut cmd + k | ctrl + k
Fastly Object Storage Import

Prerequisites

For Fastly Object Storage, the S3 Compatibility API allows you to use DuckDB's S3 support to read and write from Fastly buckets.

This requires the httpfs extension, which can be installed using the INSTALL SQL command. This only needs to be run once.

Credentials and Configuration

You will need to generate an S3 auth token and create an S3 secret in DuckDB:

CREATE SECRET my_secret (
    TYPE S3,
    KEY_ID 'AKIAIOSFODNN7EXAMPLE',
    SECRET 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY',
	URL_STYLE 'path',
    REGION 'us-east',
    ENDPOINT 'us-east.object.fastlystorage.app' -- see note below
);
  • ENDPOINT needs to point to the Fastly endpoint for the region you want to use (e.g eu-central.object.fastlystorage.app).
  • REGION must use the same region mentioned in ENDPOINT.
  • URL_STYLE needs to use path.

Querying

After setting up the Fastly Object Storage credentials, you can query the data there using DuckDB's built-in methods, such as read_csv or read_parquet:

SELECT * FROM 's3://⟨fastly_bucket_name⟩/(file).csv';
SELECT * FROM read_parquet('s3://⟨fastly_bucket_name⟩/⟨file⟩.parquet');
About this page