Provides high-performance, space-efficient probabilistic data structures—including quotient, XOR, and binary fuse filters—for fast approximate set membership testing with no false negatives and configurable false positive rates.
Maintainer(s):
rustyconover
Compatibility Matrix
This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.
| Platform | 1.4.4 | 1.4.5 | 1.5.5 | 1.5.6 |
|---|---|---|---|---|
| Linux AMD64 | ||||
| Linux ARM64 | ||||
| macOS AMD64 | ||||
| macOS ARM64 | ||||
| Windows AMD64 | ||||
| Windows ARM64 | ||||
| WebAssembly |
Installing and Loading
INSTALL bitfilters FROM community;
LOAD bitfilters;
About bitfilters
For more information regarding usage, see the documentation.
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| binary_fuse16_filter | aggregate | Creates a Binary Fuse 16-bit filter (~0.0015% false positive rate) by aggregating UBIGINT values. Returns a BLOB to probe with binary_fuse16_filter_contains. | NULL | [SELECT binary_fuse16_filter(hash(column)) FROM table] |
| binary_fuse16_filter_contains | scalar | Tests if a Binary Fuse 16-bit filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). | NULL | [SELECT binary_fuse16_filter_contains(filter, 42) FROM table] |
| binary_fuse8_filter | aggregate | Creates a Binary Fuse 8-bit filter (~0.4% false positive rate) by aggregating UBIGINT values. Returns a BLOB to probe with binary_fuse8_filter_contains. | NULL | [SELECT binary_fuse8_filter(hash(column)) FROM table] |
| binary_fuse8_filter_contains | scalar | Tests if a Binary Fuse 8-bit filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). | NULL | [SELECT binary_fuse8_filter_contains(filter, 42) FROM table] |
| bitfilters_duckdb_bloom_filter_create | aggregate | Creates a DuckDB-compatible bloom filter by aggregating pre-hashed UBIGINT values. Use bitfilters_duckdb_hash() to hash values first. The version and num_sectors (power of 2) are constants. Returns a BLOB that can be probed with bitfilters_duckdb_bloom_filter_probe. | NULL | [SELECT bitfilters_duckdb_bloom_filter_create('v1.5.1', 16384, bitfilters_duckdb_hash('v1.5.1', key)) FROM tbl] |
| bitfilters_duckdb_bloom_filter_probe | scalar | Probes a DuckDB-internal bloom filter (serialized as BLOB) for a value. The version parameter specifies which DuckDB hash algorithm to use. The filter blob and version are constant-folded at bind time. Multiple value arguments are combined for multi-key bloom filter probes. | NULL | [SELECT bitfilters_duckdb_bloom_filter_probe('v1.5.1', filter_blob, key_col) FROM tbl, SELECT bitfilters_duckdb_bloom_filter_probe('v1.5.1', filter_blob, col1, col2) FROM tbl] |
| bitfilters_duckdb_hash | scalar | Computes the DuckDB-internal hash value for the given DuckDB version. The hash matches DuckDB's built-in hash() function for that version. Multiple values are combined using the version's CombineHash algorithm. Supported versions: v1.4.0-v1.4.4, v1.5.0, v1.5.1, v1.6.0. | NULL | [SELECT bitfilters_duckdb_hash('v1.5.1', 42), SELECT bitfilters_duckdb_hash('v1.5.1', col1, col2) FROM tbl] |
| quotient_filter | aggregate | Creates a Quotient filter by aggregating values or by merging other Quotient filters. q is the number of quotient bits (controls capacity, ~2^q slots) and r is the number of remainder bits (controls false positive rate, ~2^-r). q + r must be <= 64. | NULL | [SELECT quotient_filter(16, 8, column) FROM table] |
| quotient_filter | aggregate | NULL | NULL | |
| quotient_filter_contains | scalar | Tests if a Quotient filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). | NULL | [SELECT quotient_filter_contains(filter, 42) FROM table] |
| xor16_filter | aggregate | Creates an Xor16 filter (16-bit fingerprints, ~0.0015% false positive rate) by aggregating UBIGINT values. Returns a BLOB to probe with xor16_filter_contains. | NULL | [SELECT xor16_filter(hash(column)) FROM table] |
| xor16_filter_contains | scalar | Tests if an Xor16 filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). | NULL | [SELECT xor16_filter_contains(filter, 42) FROM table] |
| xor8_filter | aggregate | Creates an Xor8 filter (8-bit fingerprints, ~0.4% false positive rate) by aggregating UBIGINT values. Returns a BLOB to probe with xor8_filter_contains. | NULL | [SELECT xor8_filter(hash(column)) FROM table] |
| xor8_filter_contains | scalar | Tests if an Xor8 filter may contain a value. Returns true if the value might be in the set (with possible false positives), or false if the value is definitely not in the set (no false negatives). | NULL | [SELECT xor8_filter_contains(filter, 42) FROM table] |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
This extension does not add any settings.