Downloading Extensions Directly from S3

Downloading an extension directly could be helpful when building a lambda service or container that uses DuckDB. DuckDB extensions are stored in public S3 buckets, but the directory structure of those buckets is not searchable. As a result, a direct URL to the file must be used. To download an extension file directly, use the following format:

http://extensions.duckdb.org/v{duckdb_version}/{platform_name}/{extension_name}.duckdb_extension.gz

For example:

http://extensions.duckdb.org/v0.10.2/windows_amd64/json.duckdb_extension.gz

Platforms

Extension binaries must be built for each platform. We distribute pre-built binaries for several platforms (see below). For platforms where packages for certain extensions are not available, users can build them from source and install the resulting binaries manually.

All official extensions are distributed for the following platforms.

For some Linux ARM distributions (e.g., Python), two different binaries are distributed. These target either the linux_arm64 or linux_arm64_gcc4 platforms. Note that extension binaries are distributed for the first, but not the second. Effectively that means that on these platforms your glibc version needs to be 2.28 or higher to use the distributed extension binaries.

Some extensions are distributed for the following platforms:

• windows_amd64_rtools
• wasm_eh and wasm_mvp (see DuckDB-Wasm’s extensions)

For platforms outside the ones listed above, we do not officially distribute extensions (e.g., linux_arm64_gcc4, windows_amd64_mingw).

Using a Custom Extension Repository

To load extensions from a custom extension repository, set the following configuration option.

Local Files

SET custom_extension_repository = 'path/to/folder';

This assumes the pointed folder has a structure similar to:

folder
└── 0fd6fb9198
    └── osx_arm64
        ├── autocomplete.duckdb_extension
        ├── httpfs.duckdb_extension
        ├── icu.duckdb_extension
        ├── inet.duckdb_extension
        ├── json.duckdb_extension
        ├── parquet.duckdb_extension
        ├── tpcds.duckdb_extension
        ├── tpcds.duckdb_extension
        └── tpch.duckdb_extension

With at the first level the DuckDB version, at the second the DuckDB platform, and then extensions as name.duckdb_extension.

INSTALL icu;

The execution of this statement will first look icu.duckdb_extension.gz, then icu.duckdb_extension the folder’s file structure. If it finds either of the extension binaries, it will install the extension to the location specified by the extension_directory option (which defaults to ~/.duckdb/extensions).

Remote File over http

SET custom_extension_repository = 'http://nightly-extensions.duckdb.org';

They work the same as local ones, and expect a similar folder structure.

Remote Files over https or s3 Protocol

SET custom_extension_repository = 's3://bucket/your-repository-name/';

Remote extension repositories act similarly to local ones, as in the file structure should be the same and either gzipped or non-gzipped file are supported.

Only special case here is that httpfs extension should be available locally. You can get it for example doing:

RESET custom_extension_repository;
INSTALL httpfs;

That will install the official httpfs extension locally.

This is since httpfs extension will be needed to actually access remote encrypted files.

INSTALL x FROM y

You can also use the INSTALL command’s FROM clause to specify the path of the custom extension repository. For example:

FORCE INSTALL azure FROM 'http://nightly-extensions.duckdb.org';

This will force install the azure extension from the specified URL.

Loading and Installing an Extension from Explicit Paths

Installing Extensions from an Explicit Path

INSTALL can be used with the path to either a .duckdb_extension file. .duckdb_extension.gz files need to be decompressed before issuing INSTALL name.duckdb_extension;.

For example, if the file was available into the same directory as where DuckDB is being executed, you can install it as follows:

-- uncompressed file
INSTALL 'path/to/httpfs.duckdb_extension';

It is also possible to specify remote paths.

Force Installing Extensions

When DuckDB installs an extension, it is copied to a local directory to be cached, avoiding any network traffic. Any subsequent calls to INSTALL extension_name will use the local version instead of downloading the extension again. To force re-downloading the extension, run:

FORCE INSTALL extension_name;

For more details, see the Versioning of Extensions page.

Loading Extension from a Path

LOAD can be used with the path to a .duckdb_extension. For example, if the file was available at the (relative) path path/to/httpfs.duckdb_extension, you can load it as follows:

-- uncompressed file
LOAD 'path/to/httpfs.duckdb_extension';

This will skip any currently installed file in the specifed path.

Using remote paths for compressed files is currently not possible.

Building and Installing Extensions

For building and installing extensions from source, see the building guide.

Statically Linking Extensions

To statically link extensions, follow the developer documentation’s “Using extension config files” section.