Search Shortcut cmd + k | ctrl + k
bitfilters
Downloads 1.2kthis week
GitHub stars 5
Extension repository on GitHub
Extension descriptor (YAML)

Provides high-performance, space-efficient probabilistic data structures—including quotient, XOR, and binary fuse filters—for fast approximate set membership testing with no false negatives and configurable false positive rates.

Maintainer(s): rustyconover

Installing and Loading 

INSTALL bitfilters FROM community;
LOAD bitfilters;

About bitfilters

For more information regarding usage, see the documentation.

Added Functions

function_name function_type description comment examples
binary_fuse16_filter aggregate Creates a Binary Fuse 16-bit filter with ~0.0015% false positive rate. NULL [SELECT binary_fuse16_filter(hash(column)) FROM table]
binary_fuse16_filter_contains scalar Tests if a BinaryFuse16 filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). NULL [SELECT binary_fuse16_filter_contains(filter, 42) FROM table]
binary_fuse8_filter aggregate Creates a Binary Fuse 8-bit filter with ~0.4% false positive rate. NULL [SELECT binary_fuse8_filter(hash(column)) FROM table]
binary_fuse8_filter_contains scalar Tests if a BinaryFuse8 filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). NULL [SELECT binary_fuse8_filter_contains(filter, 42) FROM table]
quotient_filter aggregate Creates a Quotient filter by aggregating values or by merging other Quotient filters. Takes q and r as number of bits. NULL [SELECT quotient_filter(16, 8, column) FROM table]
quotient_filter_contains scalar Tests if a Quotient filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). NULL [SELECT quotient_filter_contains(filter, 42) FROM table]
xor16_filter aggregate Creates a Xor16 filter with ~0.0015% false positive rate. NULL [SELECT xor16_filter(hash(column)) FROM table]
xor16_filter_contains scalar Tests if a Xor16 filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). NULL [SELECT xor16_filter_contains(filter, 42) FROM table]
xor8_filter aggregate Creates a Xor8 filter with ~0.4% false positive rate. NULL [SELECT xor8_filter(hash(column)) FROM table]
xor8_filter_contains scalar Tests if a Xor8 filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). NULL [SELECT xor8_filter_contains(filter, 42) FROM table]

Added Settings

name description input_type scope aliases
auto_fallback_to_full_download Allows automatically falling back to full file downloads when possible. BOOLEAN GLOBAL []
ca_cert_file Path to a custom certificate file for self-signed certificates. VARCHAR GLOBAL []
enable_curl_server_cert_verification Enable server side certificate verification for CURL backend. BOOLEAN GLOBAL []
enable_server_cert_verification Enable server side certificate verification. BOOLEAN GLOBAL []
force_download Forces upfront download of file BOOLEAN GLOBAL []
hf_max_per_page Debug option to limit number of items returned in list requests UBIGINT GLOBAL []
http_keep_alive Keep alive connections. Setting this to false can help when running into connection failures BOOLEAN GLOBAL []
http_retries HTTP retries on I/O error UBIGINT GLOBAL []
http_retry_backoff Backoff factor for exponentially increasing retry wait time FLOAT GLOBAL []
http_retry_wait_ms Time between retries UBIGINT GLOBAL []
http_timeout HTTP timeout read/write/connection/retry (in seconds) UBIGINT GLOBAL []
httpfs_client_implementation Select which is the HTTPUtil implementation to be used VARCHAR GLOBAL []
merge_http_secret_into_s3_request Merges http secret params into S3 requests BOOLEAN GLOBAL []
s3_access_key_id S3 Access Key ID VARCHAR GLOBAL []
s3_endpoint S3 Endpoint VARCHAR GLOBAL []
s3_kms_key_id S3 KMS Key ID VARCHAR GLOBAL []
s3_region S3 Region VARCHAR GLOBAL []
s3_requester_pays S3 use requester pays mode BOOLEAN GLOBAL []
s3_secret_access_key S3 Access Key VARCHAR GLOBAL []
s3_session_token S3 Session Token VARCHAR GLOBAL []
s3_uploader_max_filesize S3 Uploader max filesize (between 50GB and 5TB) VARCHAR GLOBAL []
s3_uploader_max_parts_per_file S3 Uploader max parts per file (between 1 and 10000) UBIGINT GLOBAL []
s3_uploader_thread_limit S3 Uploader global thread limit UBIGINT GLOBAL []
s3_url_compatibility_mode Disable Globs and Query Parameters on S3 URLs BOOLEAN GLOBAL []
s3_url_style S3 URL style VARCHAR GLOBAL []
s3_use_ssl S3 use SSL BOOLEAN GLOBAL []
unsafe_disable_etag_checks Disable checks on ETag consistency BOOLEAN GLOBAL []