loki

579 downloads this week 0 stars Extension descriptor

Query Grafana Loki from DuckDB in plain SQL, with label/time/line-filter pushdown into LogQL

Compatibility Matrix

This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.

Installing and Loading

INSTALL loki FROM community;
LOAD loki;

Example

-- Store the Loki endpoint and credentials once with the Secret Manager.
CREATE SECRET my_loki (
    TYPE loki,
    ENDPOINT 'https://logs-prod.example.net',
    TOKEN 'glc_...'            -- or USERNAME + PASSWORD for basic auth
);

-- Query logs in plain SQL. WHERE predicates on promoted label columns, the
-- timestamp and the log line are translated into a LogQL stream selector,
-- start/end bounds and line filters; anything else DuckDB applies as a residual.
SELECT timestamp, level, line
FROM loki(selector := '{job="api"}', labels := ['level'], secret := 'my_loki')
WHERE level = 'error'
  AND timestamp >= now() - INTERVAL 2 HOUR
  AND line LIKE '%timeout%';

-- Or send raw LogQL through untouched with loki_scan():
SELECT * FROM loki_scan('{job="api"} |= `timeout`', secret := 'my_loki');

-- Or ATTACH the whole instance and reach logs by name. Loki requires a stream
-- selector, so always filter on at least one label column.
ATTACH 'my_loki' AS lk (TYPE loki);
SELECT timestamp, line FROM lk.public.logs WHERE job = 'api';

About loki

loki turns a Grafana Loki instance into a first-class, queryable DuckDB data source, so you can join your logs against Parquet, CSV, Postgres, or anything else DuckDB reads — without ever hand-writing an HTTP request.

• loki(selector := ...) — pushdown-driven. SQL WHERE filters on promoted label columns, timestamp, and line are translated into the LogQL stream selector, start/end time bounds, and line filters. Untranslatable predicates are left as residuals for DuckDB to apply, so results are always correct.
• loki_scan(query := ...) — a raw-LogQL escape hatch for full control.
• loki_labels() / loki_label_values() / loki_series() — label discovery helpers so the source is explorable and autocompletes well.
• ATTACH '...' AS lk (TYPE loki) — Loki as a read-only attached database: lk.<tenant>.logs (plus labels and series) with the same pushdown, label columns auto-discovered at attach time and refreshable via PRAGMA loki_refresh_labels. Because Loki refuses a query without a stream selector, a scan of logs must carry a WHERE on at least one label column.

Connection and auth (bearer token, basic auth, X-Scope-OrgID tenant, or arbitrary headers) come from DuckDB's Secret Manager with per-call overrides. A CREATE SECRET (TYPE loki, PROVIDER env) seeds the connection from the standard logcli environment variables (LOKI_ADDR, LOKI_BEARER_TOKEN, LOKI_USERNAME/LOKI_PASSWORD, LOKI_ORG_ID), with inline options overriding the environment. Results page automatically over Loki's per-request entry cap, and projection pushdown skips materialising the columns a query doesn't select. Timestamps keep Loki's nanosecond resolution (TIMESTAMP_NS). See the repository for full documentation.

Added Functions

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings