decide_onnx

n/a downloads this week 4 stars Extension descriptor

Classify rows, route messages and answer yes/no questions in SQL with the vela3-fast decision model, on CPU via ONNX Runtime

Compatibility Matrix

This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.

Installing and Loading

INSTALL decide_onnx FROM community;
LOAD decide_onnx;

Example

INSTALL decide_onnx FROM community;
LOAD decide_onnx;

-- vela3-fast (1.5 GB, an ONNX export) is downloaded from assets.altertable.ai and cached on first use.
CREATE TABLE tickets AS FROM (VALUES
    ('Broken mug', 'The order arrived broken. I want my money back.'),
    ('Double charge', 'I was charged twice this month.')
) t(subject, message);

-- Route every ticket and flag refund requests: an answer, its confidence and all probabilities per question.
FROM tickets DECIDE (
    CHOOSE 'Which team should handle this customer message?' AMONG (
        billing  'Billing: charges, refunds, invoices',
        shipping 'Shipping: delivery, lost or damaged packages',
        tech     'Tech support: bugs, crashes, login problems') AS team,
    ASK 'Is the customer asking for a refund or an exchange?' AS refund
    GIVEN subject, message
);

-- The same decisions as functions.
SELECT decision_choose(message, 'Which team?', ['billing', 'shipping', 'tech support']).choice,
       decision_yes_no(message, 'Is the customer asking for a refund?')
FROM tickets;

About decide_onnx

decide_onnx runs the decision backbone of vela3-fast, a 380M-parameter encoder of the vLLM Semantic Router Vela 3.0 family, inside DuckDB. You give it a state, a question and options; it picks an option and gives a probability for each, or gives P(yes) for a yes/no question, in one encoder pass. There is no prompt engineering of an LLM, no Python and no GPU: the network runs on CPU through a statically linked ONNX Runtime.

• FROM t DECIDE (CHOOSE question AMONG (key 'description', ...) AS name, ASK question AS name, ... GIVEN columns) adds name, name_confidence and name_probabilities (a MAP of option to probability) per question; filter on them with an ordinary WHERE.
• decision_choose(state, question, options) returns the choice, its confidence and every option's probability; options are a list of texts, or a MAP or STRUCT of key to description. decision_yes_no(state, question) returns P(yes). Both also take the clause's words: decision_choose('Which team?' AMONG (billing, tech) GIVEN subject, message).
• A STRUCT, MAP or JSON state is read as compact JSON, the way the reference reads a JSON state.
• decide_onnx_models() lists the model files and whether they are cached. Set decide_onnx_model_path to use a local copy offline.

Send LOAD decide_onnx and the first DECIDE as two statements. A connection that is already open picks up the new syntax when its current query ends.

The tokenizer is a C++ port of the Hugging Face one and matches it token for token; the probabilities match the published PyTorch code to 1e-4. Rows run in parallel on DuckDB's threads and repeated decisions are scored once.

The extension is MIT licensed. The vela3-fast weights are Apache-2.0. The model's probabilities are not calibrated; do not use it for high-risk decisions about people without independent validation.

Added Functions

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings