Provides IO observability to filesystem
Installing and Loading
INSTALL observefs FROM community;
LOAD observefs;
Example
COPY (SELECT observefs_get_profile()) TO '/tmp/output.txt';
About observefs
This extension provides observability to duckdb filesystems. It supports a few key features:
- 100% compatible with duckdb httpfs
- Record latency for most of the IO operations (open, read, write, etc)
- Provides both process-wise and bucket-wise latency stats (including histogram and quantile estimation) to all read operations
- Provides cache access insight to duckdb external file cache
- Allows registering ANY duckdb compatible filesystems (i.e., azure filesystem)
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| observefs_clear | scalar | NULL | NULL | |
| observefs_clear_external_file_cache_access_record | scalar | NULL | NULL | |
| observefs_external_file_cache_access_record | table | NULL | NULL | |
| observefs_get_profile | scalar | NULL | NULL | |
| observefs_list_registered_filesystems | table | NULL | NULL | |
| observefs_wrap_filesystem | scalar | NULL | NULL |
Added Settings
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| observefs_enable_external_file_cache_stats | Whether to enable stats record for external file cache. | BOOLEAN | GLOBAL | [] |