Remote storage filesystem for DuckDB via Apache OpenDAL — S3, GCS, Azure, Hugging Face, WebDAV, SFTP, and more, with DuckDB secret-based auth.
Maintainer(s):
dentiny
Installing and Loading
INSTALL duckdb_opendalfs FROM community;
LOAD duckdb_opendalfs;
Example
FORCE INSTALL duckdb_opendalfs FROM community;
LOAD duckdb_opendalfs;
CREATE SECRET production_s3 (
TYPE opendal_s3,
SCOPE 's3://analytics',
ACCESS_KEY_ID 'access-key',
SECRET_ACCESS_KEY 'secret-key',
REGION 'us-east-1'
);
SELECT * FROM read_parquet('s3://analytics/events.parquet');
About duckdb_opendalfs
duckdb_opendalfs is an extension which supports many backends (S3, GCS, Azure Blob, Hugging Face, etc)
and secret management via opendal_<backend> secret types.
Added Functions
This extension does not add any functions.
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| opendal_io_timeout_ms | OpenDAL I/O-operation timeout in milliseconds | UBIGINT | GLOBAL | [] |
| opendal_retry_factor | OpenDAL exponential retry factor | DOUBLE | GLOBAL | [] |
| opendal_retry_jitter | Add jitter to OpenDAL retry delays | BOOLEAN | GLOBAL | [] |
| opendal_retry_layer_enabled | Enable the OpenDAL retry layer | BOOLEAN | GLOBAL | [] |
| opendal_retry_max_delay_ms | Maximum OpenDAL retry delay in milliseconds | UBIGINT | GLOBAL | [] |
| opendal_retry_max_times | Maximum number of OpenDAL retry backoff delays | UBIGINT | GLOBAL | [] |
| opendal_retry_min_delay_ms | Minimum OpenDAL retry delay in milliseconds | UBIGINT | GLOBAL | [] |
| opendal_timeout_layer_enabled | Enable the OpenDAL timeout layer | BOOLEAN | GLOBAL | [] |
| opendal_timeout_ms | OpenDAL control-operation timeout in milliseconds | UBIGINT | GLOBAL | [] |