Allows reading and writing files over WebDAV protocol
Installing and Loading
INSTALL webdavfs FROM community;
LOAD webdavfs;
Example
-- Load the extension
INSTALL webdavfs FROM community;
LOAD webdavfs;
-- Authenticate with any WebDAV server eg nextcloud or owncloud
CREATE SECRET my_storagebox (
TYPE WEBDAV,
USERNAME 'u123456',
PASSWORD 'password',
SCOPE 'webdav://webdav-server.example.com/'
);
-- or with Hetzner specific Storage Box
CREATE SECRET my_storagebox (
TYPE WEBDAV,
USERNAME 'u123456',
PASSWORD 'password',
SCOPE 'storagebox://u123456'
);
-- Convert local csv into parquet and upload it using WebDAV
COPY (
FROM 'local.csv'
) TO 'storagebox://u123456/remote.parquet';
-- Read The uploaded parquet file using WebDAV
SELECT * FROM 'storagebox://u123456/remote.parquet';
About webdavfs
DuckDB WebDAVfs extension enables seamless integration with WebDAV servers, allowing users to read from and write to remote file systems directly within DuckDB. This extension supports authentication mechanisms compatible with various WebDAV services, including Hetzner Storage Boxes.
WebDAV is nice because it builds on top of HTTP/HTTPS protocols. This enables DuckDB to leverage http 1.1 range feature to only download parts of files when needed, making it efficient for working with large datasets stored remotely.
Hetzner Storage Boxes are a popular choice for WebDAV storage being one of the cheapest storage options available.
See: https://github.com/midwork-finds-jobs/webdavfs/blob/main/README.md for more examples and details.
Added Settings
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| webdav_debug_logging | Enable debug logging for WebDAV operations | BOOLEAN | GLOBAL | [] |
| webdav_max_retries | Maximum number of retries for failed WebDAV operations | BIGINT | GLOBAL | [] |
| webdav_streaming_threshold_mb | File size threshold in MB for streaming uploads (files larger than this are streamed from disk) | BIGINT | GLOBAL | [] |