hedged_request_fs

Search Shortcut cmd + k | ctrl + k

Documentation

hedged_request_fs

Downloads 296this week

GitHub stars 1

Extension repository on GitHub

Extension descriptor (YAML)

Provides hedged request for filesystem operations

Maintainer(s): dentiny

INSTALL hedged_request_fs FROM community;
LOAD hedged_request_fs;

SELECT * FROM hedged_fs_list_filesystems();
SELECT hedged_fs_wrap('S3FileSystem');

This extension enhances DuckDB filesystem capability by hedged request to cut down tail latency.

It allows users to wrap an existing filesystem, which achieves hedged request with no user code change.
It supports all metadata and directory operations, the slow ones of which will be attempted hedged request to cut down tail latency.
Users are able to tune hedged requests trigger thresholds and max attempt counts.

function_name	function_type	description	comment	examples
hedged_fs_list_filesystems	table	NULL	NULL
hedged_fs_wrap	scalar	NULL	NULL

name	description	input_type	scope	aliases
hedged_fs_directory_exists_delay_ms	Delay in milliseconds before starting hedged request for DirectoryExists operation	UBIGINT	GLOBAL	[]
hedged_fs_file_exists_delay_ms	Delay in milliseconds before starting hedged request for FileExists operation	UBIGINT	GLOBAL	[]
hedged_fs_get_file_size_delay_ms	Delay in milliseconds before starting hedged request for GetFileSize operation	UBIGINT	GLOBAL	[]
hedged_fs_get_file_type_delay_ms	Delay in milliseconds before starting hedged request for GetFileType operation	UBIGINT	GLOBAL	[]
hedged_fs_get_last_modified_time_delay_ms	Delay in milliseconds before starting hedged request for GetLastModifiedTime operation	UBIGINT	GLOBAL	[]
hedged_fs_get_version_tag_delay_ms	Delay in milliseconds before starting hedged request for GetVersionTag operation	UBIGINT	GLOBAL	[]
hedged_fs_glob_delay_ms	Delay in milliseconds before starting hedged request for Glob operation	UBIGINT	GLOBAL	[]
hedged_fs_list_files_delay_ms	Delay in milliseconds before starting hedged request for ListFiles operation	UBIGINT	GLOBAL	[]
hedged_fs_max_hedged_request_count	Maximum number of hedged requests to spawn for each operation	UBIGINT	GLOBAL	[]
hedged_fs_open_file_delay_ms	Delay in milliseconds before starting hedged request for OpenFile operation	UBIGINT	GLOBAL	[]