Downloads 659this week

GitHub stars 4

Extension repository on GitHub

Extension descriptor (YAML)

Zero-shot tabular machine learning inside DuckDB — classification, regression, synthetic-data generation and imputation with real tabular foundation models (Mitra, TabPFN v2/2.5/3, TabICL, Orion-BiX, TabFM) on ONNX Runtime, no training loop

Installing and Loading

INSTALL anofox_tabfm FROM community;
LOAD anofox_tabfm;

Example

-- Fetch a real tabular foundation model once (Mitra, Apache-2.0, ~300 MB,
-- no license gate). Cached under ~/.cache/anofox-tabfm and reused after.
INSTALL httpfs; LOAD httpfs;                       -- weights are fetched over HTTPS
CALL tabfm_download('classification', model := 'mitra');

-- Label a few rows; leave the ones you want scored as NULL. The model reads
-- the labelled rows as in-context examples and predicts the rest — no training.
CREATE TABLE iris AS SELECT * FROM VALUES
  (5.1, 3.5, 1.4, 0.2, 'setosa'),
  (4.9, 3.0, 1.4, 0.2, 'setosa'),
  (7.0, 3.2, 4.7, 1.4, 'versicolor'),
  (6.4, 3.2, 4.5, 1.5, 'versicolor'),
  (6.3, 3.3, 6.0, 2.5, 'virginica'),
  (5.8, 2.7, 5.1, 1.9, 'virginica'),
  (5.0, 3.6, 1.4, 0.2, NULL),                      -- predict me
  (6.5, 3.0, 5.8, 2.2, NULL)                       -- and me
  AS t(sepal_len, sepal_wid, petal_len, petal_wid, species);

SELECT petal_len, petal_wid, yhat AS predicted_species, yhat_score
FROM tabfm_classify('iris', 'species', model := 'mitra')
WHERE species IS NULL;

About anofox_tabfm

anofox_tabfm embeds real tabular foundation models — TabPFN-style in-context learners — into DuckDB, so tabular classification and regression become a single SQL statement. There is no training loop, no Python, and no MLOps: the model reads your labelled rows as context and predicts the rest.

Built-in models

Seven models ship in the extension and are selected with model := (or a SET anofox_tabfm_default_model once per session):

• mitra — AWS AutoGluon (Apache-2.0), no license gate, ~300 MB. A great default.
• tabpfn-v2 — Prior Labs (Apache-2.0, attribution).
• tabicl-v2 — Inria (BSD-3-Clause).
• orion-bix — Lexsi Labs (MIT), no license gate. Classification only.
• tabpfn-v2-5 — Prior Labs TabPFN 2.5. Non-commercial: the weights and their outputs may not be used for commercial or production purposes.
• tabpfn-v3 — Prior Labs TabPFN 3. Non-commercial: testing, evaluation and internal benchmarking only.
• tabfm-v1 — Google TabFM (non-commercial, gated; ~6.6 GB).

SELECT model, license, commercial FROM tabfm_list_models() reports each model's license and whether it is commercially usable — worth checking before shipping anything built on one.

Only weight-free computation graphs are bundled — no model weights are distributed with the extension. You download the weights yourself from Hugging Face into a local cache (~/.cache/anofox-tabfm), and for a gated model you first accept its license (SET anofox_tabfm_accept_hf_license = true). Repositories that Hugging Face itself gates additionally need a token, supplied with a standard DuckDB secret:

CREATE SECRET hf (TYPE http, BEARER_TOKEN 'hf_xxx', SCOPE 'https://huggingface.co');

Bring your own model

Register any compatible model entirely from SQL — no external JSON manifest. Weights can be .safetensors or a native PyTorch .ckpt (read without Python):

CALL tabfm_register_model(
  id := 'my-model',
  classification_graph   := 'model.onnx',
  classification_weights := 'model.safetensors',
  license := 'apache-2.0');

Synthetic data and imputation

The same in-context engine also runs backwards: instead of predicting one column, it models the whole table as a joint distribution and samples from it.

SELECT * FROM tabfm_generate('customers', 500);        -- 500 synthetic rows
CREATE TABLE clean AS SELECT * FROM tabfm_impute('raw');  -- fill every NULL

Generation works column by column, each column sampled conditioned on the ones already generated (the chain rule), so the relationships between columns survive — not just each column's marginal. It needs only classification weights, so it works with every model above, including classification-only ones.

tabfm_impute is the deterministic sibling: it takes the conditional best estimate rather than sampling, so continuous fills keep full precision and non-NULL cells are never modified.

On the Prior Labs breast-cancer benchmark (30 features), a classifier given only synthetic in-context examples scores 97.8% on held-out real rows against 98.3% for the real training data, preserving correlation structure at 0.97 across all 435 feature pairs. Correlations are somewhat attenuated by the quantile binning used for continuous columns — see docs/GENERATE.md in the repository for what the method does and does not preserve. It is not a differential-privacy mechanism.

Inference

Runs on ONNX Runtime, statically linked into the extension (CPU execution provider). CUDA and ROCm/MIGraphX flavors exist in the source tree for self-builds; this community build ships the portable CPU flavor.

Surface

• tabfm_classify / tabfm_regress — zero-shot predict (a single table with NULL targets, or a separate test := set)
• tabfm_generate / tabfm_impute — synthesize rows from a table's joint distribution, or fill its NULL cells
• tabfm_predict, tabfm_predict_by, tabfm_predict_agg, tabfm_predict_win
• tabfm_register_model / tabfm_unregister_model — pure-SQL model registration
• tabfm_download / tabfm_models / tabfm_list_models / tabfm_load / tabfm_unload / tabfm_remove — all accept model :=
• tabfm_devices — discover CPU/GPU execution providers
• SET anofox_tabfm_* settings (default model, license gate, cache dir, threads, device, tracing)

Full function names are anofox_tabfm_* with short tabfm_* aliases. See the project repository for the full SQL API and examples.

Added Functions

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings