Support
Search Shortcut cmd + k | ctrl + k
Search cmd+k ctrl+k
0.10 (stable)
HTTP(S) Support

With the httpfs extension, it is possible to directly query files over the HTTP(S) protocol. This works for all files supported by DuckDB or its various extensions, and provides read-only access.

SELECT *
FROM 'https://domain.tld/file.extension';

For CSV files, files will be downloaded entirely in most cases, due to the row-based nature of the format. For Parquet files, DuckDB can use a combination of the Parquet metadata and HTTP range requests to only download the parts of the file that are actually required by the query. For example, the following query will only read the Parquet metadata and the data for the column_a column:

SELECT column_a
FROM 'https://domain.tld/file.parquet';

In some cases even, no actual data needs to be read at all as they only require reading the metadata:

SELECT count(*)
FROM 'https://domain.tld/file.parquet';

Scanning multiple files over HTTP(S) is also supported:

SELECT *
FROM read_parquet([
    'https://domain.tld/file1.parquet',
    'https://domain.tld/file2.parquet'
]);

Using a Custom Certificate File

This feature is currently only available in the nightly build. It will be released in version 0.10.1.

To use the httpfs extension with a custom certificate file, set the following configuration options prior to loading the extension:

LOAD httpfs;
SET ca_cert_file = '⟨certificate_file⟩';
SET enable_server_cert_verification = true;
About this page

Last modified: 2024-04-25