httpfs with connection pool, HTTP/2 and async IO.
Maintainer(s):
dentiny
Installing and Loading
INSTALL curl_httpfs FROM community;
LOAD curl_httpfs;
Example
SELECT length(content) AS char_count FROM read_text('https://raw.githubusercontent.com/dentiny/duck-read-cache-fs/main/test/data/stock-exchanges.csv');
About curl_httpfs
This extension rewrites HTTP layer for httpfs based on libcurl and epoll, it provides a few features:
- 100% compatible with httpfs; use curl-based implementation by default, but also allows users to fallback to httplib.
- Enable HTTP/2 by default.
- Implements TCP connection pool.
- All network IO operations are performed in asynchronously.
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| curl_httpfs_clear_metrics | scalar | NULL | NULL | |
| curl_httpfs_get_httpfs_tcp_connection | table | NULL | NULL | |
| curl_httpfs_get_tcp_connection | table | NULL | NULL | |
| curl_httpfs_http_util_name | scalar | NULL | NULL |
Overloaded Functions
| function_name | function_type | description | comment | examples | |—————|—————|————-|———|———-|
Added Types
| type_name | type_size | logical_type | type_category | internal | |———–|———-:|————–|—————|———-|
Added Settings
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| curl_httpfs_client_implementation | Select the HTTPUtil implementation to be used. Supports multi_curl, curl, httplib, and default (multi_curl). |
VARCHAR | GLOBAL | [] |
| curl_httpfs_enable_verbose_logging | Turn on and off curl-based http util verbose logging. | BOOLEAN | GLOBAL | [] |