e-means clustering in SQL: mini-batch k-means with EMA updates and fitted scaling laws (Bessa et al. 2026), in memory or streamed from a query larger than memory.
Compatibility Matrix
This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.
| Platform | 1.4.4 | 1.4.5 | 1.5.5 | 1.5.6 | 2.0.0 |
|---|---|---|---|---|---|
| Linux AMD64 | |||||
| Linux ARM64 | |||||
| macOS AMD64 | |||||
| macOS ARM64 | |||||
| Windows AMD64 | |||||
| Windows ARM64 | |||||
| WebAssembly |
Installing and Loading
INSTALL emeans FROM community;
LOAD emeans;
Example
INSTALL emeans FROM community;
LOAD emeans;
-- Two unit squares; each centroid ends at the mean of its square's four corners
SELECT list_sort((emeans([[0, 0], [0, 1], [1, 0], [1, 1],
[10, 10], [10, 11], [11, 10], [11, 11]], 2)).centroids);
-- [[0.5, 0.5], [10.5, 10.5]]
-- The nearest centroid of one vector, 1-based, and the squared distance to it
SELECT emeans_assign([9, 9], [[0, 0], [10, 10]]);
-- {'label': 2, 'sq_distance': 2.0}
-- Streamed from a query, for tables larger than memory: one row per cluster
CREATE TABLE points AS
SELECT [(i % 2) * 10 + (i % 5) * 0.1, (i % 3) * 0.1]::FLOAT[2] AS v FROM range(1000) t(i);
SELECT centroid, n FROM emeans_stream('SELECT v FROM points', 2) ORDER BY centroid[1];
-- [0.2, 0.1] 500
-- [10.2, 0.0998] 500
About emeans
e-means (Bessa et al. 2026, arXiv 2611.09501) is mini-batch k-means whose centroids follow an exponential moving average of their batch means, with the batch size and step set by fitted scaling laws. The paper reports Lloyd's 25-epoch loss beaten in 1.4 to 2.6 epochs. The extension reimplements the reference in faiss/contrib/e_means; its objective is within 0.5% of faiss-cpu's.
- emeans(vectors, k [, nepochs [, seed]]): centroids, 1-based labels and the objective per pass, for FLOAT, DOUBLE or integer lists or fixed-size arrays such as FLOAT[384]. emeans_spherical clusters by direction, for embeddings compared by cosine similarity.
- emeans_stream(query, k, …): data larger than memory. It runs the query once per pass, ordered by a hash, so DuckDB's sort spills each pass, and holds only a batch and the centroids.
- emeans_assign / emeans_assign_spherical: the nearest centroid of a vector, an IVF partition key.
- emeans_agg and emeans_cluster_id(geom, k) OVER (…): clustering of POINT geometries like PostGIS's ST_ClusterKMeans, on the sphere for geographic coordinate systems; emeans_assign_point.
- emeans_recipe and emeans_trace: the batch size and step the scaling laws pick, and a run frame by frame.
Every fit can be interrupted (Ctrl-C, con.interrupt(), max_execution_time), takes its buffers from DuckDB's buffer manager, and gives the same result for the same seed on any number of threads.
Documentation: https://hugr-lab.github.io/duck-emeans/
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| emeans | scalar | NULL | NULL | |
| emeans_agg | aggregate | NULL | NULL | |
| emeans_assign | scalar | NULL | NULL | |
| emeans_assign_point | scalar | NULL | NULL | |
| emeans_assign_spherical | scalar | NULL | NULL | |
| emeans_cluster_id | window | NULL | NULL | |
| emeans_recipe | scalar | NULL | NULL | |
| emeans_spherical | scalar | NULL | NULL | |
| emeans_stream | table | NULL | NULL | |
| emeans_stream_spherical | table | NULL | NULL | |
| emeans_trace | scalar | NULL | NULL |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
| name | description | input_type | scope | aliases | typed_value |
|---|---|---|---|---|---|
| emeans_max_work | Maximum work per emeans() call, in rows x k x d summed over all passes; 0 = no limit. A running call also stops when the query is interrupted. | DOUBLE | GLOBAL | [] | 0.0 |
| emeans_threads | Threads one emeans() call may use to assign a large batch; 0 = the threads setting. The result does not depend on it. | BIGINT | GLOBAL | [] | 0 |
| temp_directory | Set the directory to which temporary files are written. | VARCHAR | GLOBAL | [] | post.db.tmp |