A DuckLake catalog in object storage, on SlateDB
Compatibility Matrix
This table covers all DuckDB releases that have not reached end-of-life, including LTS releases, and shows the 2 most recent patch releases of each. Older patch releases are not listed, but the extension may be available for them as well.
| Platform | 1.5.5 | 1.5.6 | 2.0.0 | 2.0.1 |
|---|---|---|---|---|
| Linux AMD64 | ||||
| Linux ARM64 | ||||
| macOS AMD64 | ||||
| macOS ARM64 | ||||
| Windows AMD64 | ||||
| Windows ARM64 | ||||
| WebAssembly |
Installing and Loading
INSTALL moraine FROM community;
LOAD moraine;
Example
INSTALL moraine FROM community;
LOAD moraine;
ATTACH 'ducklake:moraine:s3://bucket/lake' AS lake
(DATA_PATH 's3://bucket/lake-data/', READ_WRITE);
CREATE TABLE lake.events (id BIGINT, payload VARCHAR);
INSERT INTO lake.events VALUES (1, 'hello, bucket');
About moraine
Moraine is a DuckLake catalog backend that keeps the catalog in object
storage on SlateDB, beside the Parquet data files, so a lake needs no
catalog server: one process writes, any number read, and an idle lake
costs only storage. It bundles the DuckLake it is built for, so LOAD
moraine serves both. ATTACH 'ducklake:moraine:
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| ducklake_add_data_files | table | NULL | NULL | |
| ducklake_cleanup_old_files | table | NULL | NULL | |
| ducklake_commit | table | NULL | NULL | |
| ducklake_current_snapshot | table | NULL | NULL | |
| ducklake_delete_orphaned_files | table | NULL | NULL | |
| ducklake_delete_positions | table | NULL | NULL | |
| ducklake_expire_snapshots | table | NULL | NULL | |
| ducklake_flush_inlined_data | table | NULL | NULL | |
| ducklake_last_committed_snapshot | table | NULL | NULL | |
| ducklake_list_files | table | NULL | NULL | |
| ducklake_merge_adjacent_files | table | NULL | NULL | |
| ducklake_options | table | NULL | NULL | |
| ducklake_rewrite_data_files | table | NULL | NULL | |
| ducklake_scan | table | NULL | NULL | |
| ducklake_set_commit_message | table | NULL | NULL | |
| ducklake_set_option | table | NULL | NULL | |
| ducklake_settings | table | NULL | NULL | |
| ducklake_snapshots | table | NULL | NULL | |
| ducklake_table_changes | table_macro | NULL | NULL | |
| ducklake_table_deletions | table | NULL | NULL | |
| ducklake_table_info | table | NULL | NULL | |
| ducklake_table_insertions | table | NULL | NULL | |
| moraine_cache_status | table | NULL | NULL | |
| moraine_cache_tally | table | NULL | NULL | |
| moraine_checkpoints | table | NULL | NULL | |
| moraine_compact_store | table | NULL | NULL | |
| moraine_create_checkpoint | table | NULL | NULL | |
| moraine_delete_checkpoint | table | NULL | NULL | |
| moraine_fault_stall_next_read | table | NULL | NULL | |
| moraine_index_in | table | NULL | NULL | |
| moraine_index_lookup | table | NULL | NULL | |
| moraine_index_nulls | table | NULL | NULL | |
| moraine_index_range | table | NULL | NULL | |
| moraine_index_verify | table | NULL | NULL | |
| moraine_indexes | table | NULL | NULL | |
| moraine_maintenance | table | NULL | NULL | |
| moraine_maintenance_status | table | NULL | NULL | |
| moraine_memory_tally | table | NULL | NULL | |
| moraine_migrate | table | NULL | NULL | |
| moraine_move_wal | table | NULL | NULL | |
| moraine_object_store_tally | table | NULL | NULL | |
| moraine_raise_catalog_version | table | NULL | NULL | |
| moraine_raise_format | table | NULL | NULL | |
| moraine_rewrite_data_path | table | NULL | NULL | |
| moraine_sidecar_tally | table | NULL | NULL | |
| moraine_store_census | table | NULL | NULL | |
| murmur3_32 | scalar | NULL | NULL |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
| name | description | input_type | scope | aliases | typed_value |
|---|---|---|---|---|---|
| ducklake_default_data_inlining_row_limit | Default row limit for data inlining (0 disables inlining) | UBIGINT | GLOBAL | [] | 10 |
| ducklake_default_version | Default DuckLake version for new catalogs | VARCHAR | GLOBAL | [] | NULL |
| ducklake_max_retry_count | The maximum amount of retry attempts for a ducklake transaction | UBIGINT | GLOBAL | [] | 10 |
| ducklake_retry_backoff | Backoff factor for exponentially increasing retry wait time | DOUBLE | GLOBAL | [] | 1.5 |
| ducklake_retry_wait_ms | Time between retries | UBIGINT | GLOBAL | [] | 100 |
| ducklake_target_file_size | Target file size for insertion and compaction | VARCHAR | GLOBAL | [] | NULL |
| ducklake_write_deletion_vectors | [EXPERIMENTAL] Write Iceberg V3 deletion vectors (puffin) instead of positional delete files (parquet) | BOOLEAN | GLOBAL | [] | false |
| moraine_cache_auxiliary_percent | Share of moraine_cache_memory for parsed Parquet metadata, capped at 50 | UBIGINT | GLOBAL | [] | 0 |
| moraine_cache_dir | Local directory for the block cache's disk tier | VARCHAR | GLOBAL | [] | |
| moraine_cache_memory | Process-wide byte cap on the in-memory block cache | VARCHAR | GLOBAL | [] | |
| moraine_cache_size | Byte cap on the block cache's disk tier, under moraine_cache_dir | VARCHAR | GLOBAL | [] | |
| moraine_commit_memory | One commit's batch | VARCHAR | GLOBAL | [] | |
| moraine_host_headroom | Memory left unclaimed on the host; a quarter of moraine_host_memory | VARCHAR | GLOBAL | [] | |
| moraine_host_memory | The host's memory, checked against the budgets before attach | VARCHAR | GLOBAL | [] | |
| moraine_staging_memory | Rows one transaction may stage | VARCHAR | GLOBAL | [] | |
| moraine_store_memory | SlateDB's unflushed writes per writer | VARCHAR | GLOBAL | [] | |
| moraine_summary_memory | One file's row-summary build | VARCHAR | GLOBAL | [] | |
| moraine_summary_scan_threads | Maximum concurrent located-scan read units | UBIGINT | GLOBAL | [] | 4 |
| moraine_working_memory | Process-wide allowance for live working allocations | VARCHAR | GLOBAL | [] | |
| temp_directory | Set the directory to which temporary files are written. | VARCHAR | GLOBAL | [] | post.db.tmp |