Search Shortcut cmd + k | ctrl + k

e-means clustering in SQL: mini-batch k-means with EMA updates and fitted scaling laws (Bessa et al. 2026), in memory or streamed from a query larger than memory.

Maintainer(s): oluies

Compatibility Matrix

This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.

Platform 1.4.4 1.4.5 1.5.5 1.5.6 2.0.0
Linux AMD64
Linux ARM64
macOS AMD64
macOS ARM64
Windows AMD64
Windows ARM64
WebAssembly

Installing and Loading

INSTALL emeans FROM community;
LOAD emeans;

Example

INSTALL emeans FROM community;
LOAD emeans;

-- Two unit squares; each centroid ends at the mean of its square's four corners
SELECT list_sort((emeans([[0, 0], [0, 1], [1, 0], [1, 1],
                          [10, 10], [10, 11], [11, 10], [11, 11]], 2)).centroids);
-- [[0.5, 0.5], [10.5, 10.5]]

-- The nearest centroid of one vector, 1-based, and the squared distance to it
SELECT emeans_assign([9, 9], [[0, 0], [10, 10]]);
-- {'label': 2, 'sq_distance': 2.0}

-- Streamed from a query, for tables larger than memory: one row per cluster
CREATE TABLE points AS
SELECT [(i % 2) * 10 + (i % 5) * 0.1, (i % 3) * 0.1]::FLOAT[2] AS v FROM range(1000) t(i);
SELECT centroid, n FROM emeans_stream('SELECT v FROM points', 2) ORDER BY centroid[1];
-- [0.2, 0.1]      500
-- [10.2, 0.0998]  500

About emeans

e-means (Bessa et al. 2026, arXiv 2611.09501) is mini-batch k-means whose centroids follow an exponential moving average of their batch means, with the batch size and step set by fitted scaling laws. The paper reports Lloyd's 25-epoch loss beaten in 1.4 to 2.6 epochs. The extension reimplements the reference in faiss/contrib/e_means; its objective is within 0.5% of faiss-cpu's.

  • emeans(vectors, k [, nepochs [, seed]]): centroids, 1-based labels and the objective per pass, for FLOAT, DOUBLE or integer lists or fixed-size arrays such as FLOAT[384]. emeans_spherical clusters by direction, for embeddings compared by cosine similarity.
  • emeans_stream(query, k, …): data larger than memory. It runs the query once per pass, ordered by a hash, so DuckDB's sort spills each pass, and holds only a batch and the centroids.
  • emeans_assign / emeans_assign_spherical: the nearest centroid of a vector, an IVF partition key.
  • emeans_agg and emeans_cluster_id(geom, k) OVER (…): clustering of POINT geometries like PostGIS's ST_ClusterKMeans, on the sphere for geographic coordinate systems; emeans_assign_point.
  • emeans_recipe and emeans_trace: the batch size and step the scaling laws pick, and a run frame by frame.

Every fit can be interrupted (Ctrl-C, con.interrupt(), max_execution_time), takes its buffers from DuckDB's buffer manager, and gives the same result for the same seed on any number of threads.

Documentation: https://hugr-lab.github.io/duck-emeans/

Added Functions

function_name function_type description comment examples
emeans scalar NULL NULL  
emeans_agg aggregate NULL NULL  
emeans_assign scalar NULL NULL  
emeans_assign_point scalar NULL NULL  
emeans_assign_spherical scalar NULL NULL  
emeans_cluster_id window NULL NULL  
emeans_recipe scalar NULL NULL  
emeans_spherical scalar NULL NULL  
emeans_stream table NULL NULL  
emeans_stream_spherical table NULL NULL  
emeans_trace scalar NULL NULL  

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings

name description input_type scope aliases typed_value
emeans_max_work Maximum work per emeans() call, in rows x k x d summed over all passes; 0 = no limit. A running call also stops when the query is interrupted. DOUBLE GLOBAL [] 0.0
emeans_threads Threads one emeans() call may use to assign a large batch; 0 = the threads setting. The result does not depend on it. BIGINT GLOBAL [] 0
temp_directory Set the directory to which temporary files are written. VARCHAR GLOBAL [] post.db.tmp
© 2026 DuckDB Foundation, Amsterdam NL
DuckDB Home Code of Conduct Trademark Use Blog