Downloads 388this week

GitHub stars 4

Extension descriptor (YAML)

A DuckDB extension for reading Apache ORC files, written in pure Rust. Supports all ORC primitives, struct, list, map types and all compression codecs (Zlib, Snappy, LZO, LZ4, ZSTD).

Installing and Loading

INSTALL orc FROM community;
LOAD orc;

Example

-- Install and use the ORC extension
INSTALL orc FROM community;
LOAD orc;

-- Read a single ORC file
SELECT * FROM read_orc('data.orc');

-- Read multiple ORC files with a glob pattern
SELECT * FROM read_orc('sales/*.orc');

-- Filter and aggregate (DuckDB applies filters post-scan)
SELECT region, SUM(amount)
FROM read_orc('sales.orc')
WHERE year = 2024
GROUP BY region;

-- Column projection is pushed down to the ORC reader
SELECT col1, col2 FROM read_orc('big_table.orc');

About orc

Built on top of orc-rust, a native Rust ORC reader that outputs Arrow record batches.

Supported ORC Features

• Data types: all ORC primitives, struct, list, map
• Encodings: all ORC encodings (direct, dictionary, RLE, etc.)
• Compression: Zlib, Snappy, LZO, LZ4, ZSTD
• Projection pushdown: column pruning via DuckDB's optimizer
• Multi-file globs: read_orc('*.orc') — all files must have identical schema
• Streaming: files of any size, read in 2048-row batches

Known Limitations

• Write support: COPY tbl TO 'file.orc' (FORMAT orc) writes with ZSTD compression. Supported types: bool, int8-64, float32/64, date, timestamp, varchar, blob. List/Struct/Map/Decimal rejected cleanly (upstream orc-rust writer doesn't encode them yet).
• Write column names: output columns are column_N — the DuckDB C API COPY bind does not expose source column names.
• Parallel scan: multi-stripe files read in parallel across threads (caps at CPU count).
• No filter pushdown: blocked by missing DuckDB C API (duckdb_table_function_supports_filter_pushdown not exposed)
• No union type: not supported by orc-rust upstream

Added Functions

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings

This extension does not add any settings.