- Installation
- What's New in the Docs for 2.0
- Documentation
- Getting Started
- Connect
- Data Import and Export
- Overview
- Data Sources
- CSV Files
- IO
- JSON Files
- Overview
- Creating JSON
- Loading JSON
- Writing JSON
- JSON Type
- JSON Functions
- Format Settings
- Installing and Loading
- SQL to / from JSON
- Caveats
- Multiple Files
- Parquet Files
- Partitioning
- Appender
- INSERT Statements
- Lakehouse Formats
- Client APIs
- Overview
- ADBC
- C
- Overview
- Startup
- Configuration
- Query
- Data Chunks
- Vectors
- Values
- Types
- Prepared Statements
- Appender
- Table Functions
- Replacement Scans
- API Reference
- C++
- CLI
- Overview
- Arguments
- Dot Commands
- Output Formats
- Agent Mode
- Editing
- Friendly CLI
- Safe Mode
- Autocomplete
- Syntax Highlighting
- Known Issues
- Go
- Overview
- Connect
- Import Data
- Run Queries
- Handle Results
- Write User Defined Functions
- Profile and Monitor
- Troubleshoot
- Java (JDBC)
- Overview
- Connect
- Import Data
- Run Queries
- Handle Results
- Write User Defined Functions
- Profile and Monitor
- Deploy as Native Image
- Troubleshoot
- Node.js (Neo)
- ODBC
- Python
- Overview
- Data Ingestion
- Appender
- Conversion between DuckDB and Python
- DB API
- Relational API
- Function API
- Types API
- Expression API
- Spark API
- API Reference
- Known Python Issues
- Migrating to 2.0
- R
- Rust
- Overview
- Connect
- Import Data
- Run Queries
- Handle Results
- Write User Defined Functions
- Profile and Monitor
- Troubleshoot
- Wasm
- Tertiary Clients
- SQL
- Introduction
- Statements
- Overview
- ANALYZE
- ALTER DATABASE
- ALTER TABLE
- ALTER VIEW
- ATTACH and DETACH
- CALL
- CHECKPOINT
- COMMENT ON
- CONNECT and DISCONNECT
- COPY
- CREATE INDEX
- CREATE MACRO
- CREATE SCHEMA
- CREATE SECRET
- CREATE SEQUENCE
- CREATE TABLE
- CREATE VIEW
- CREATE TRIGGER
- CREATE TYPE
- DELETE
- DESCRIBE
- DROP
- EXPORT and IMPORT DATABASE
- INSERT
- LOAD / INSTALL
- MERGE INTO
- PIVOT
- Profiling
- PREPARE, EXECUTE, and DEALLOCATE
- SELECT
- SET / RESET
- SET VARIABLE
- SHOW and SHOW DATABASES
- SUMMARIZE
- Transaction Management
- UNPIVOT
- UPDATE
- UPDATE EXTENSIONS
- USE
- VACUUM
- Query Syntax
- SELECT
- FROM and JOIN
- WHERE
- GROUP BY
- GROUPING SETS
- HAVING
- ORDER BY
- LIMIT and OFFSET
- SAMPLE
- Unnesting
- WITH
- WINDOW
- QUALIFY
- VALUES
- FILTER
- Set Operations
- Prepared Statements
- Data Types
- Overview
- Array
- Bitstring
- Blob
- Boolean
- Date
- Enum
- Geometry
- Interval
- List
- Literal Types
- Map
- NULL Values
- Numeric
- Struct
- Text
- Time
- Timestamp
- Time Zones
- Union
- Typecasting
- Variant
- Expressions
- Overview
- CASE Expression
- Casting
- Collations
- Comparisons
- IN Operator
- Logical Operators
- Star Expression
- Subqueries
- TRY
- Functions
- Overview
- Aggregate Functions
- Array Functions
- Bitstring Functions
- Blob Functions
- Date Format Functions
- Date Functions
- Date Part Functions
- Enum Functions
- Geometry Functions
- Interval Functions
- Lambda Functions
- List Functions
- Map Functions
- Nested Functions
- Numeric Functions
- Pattern Matching
- Regular Expressions
- Struct Functions
- Text Functions
- Time Functions
- Timestamp Functions
- Timestamp with Time Zone Functions
- Union Functions
- Utility Functions
- Variant Functions
- Window Functions
- Constraints
- Indexes
- Meta Queries
- DuckDB's SQL Dialect
- Overview
- Indexing
- Friendly SQL
- Keywords and Identifiers
- Order Preservation
- PostgreSQL Compatibility
- SQL Quirks
- PEG Parser
- Samples
- Configuration
- Extensions
- Overview
- Installing Extensions
- Advanced Installation Methods
- Window Function Extensions
- Distributing Extensions
- Extension Repositories
- Community Extensions
- Versioning of Extensions
- Troubleshooting of Extensions
- Core Extensions
- Overview
- AutoComplete
- Avro
- AWS
- Azure
- Delta
- DuckLake
- Encodings
- Excel
- Full Text Search
- httpfs (HTTP and S3)
- Overview
- HTTP(S) Support
- Hugging Face
- S3 API Support
- S3 and AWS Authentication
- Legacy Authentication Scheme for S3 API
- Iceberg
- ICU
- inet
- Lance
- Legacy Parser
- MotherDuck
- MySQL
- ODBC
- Quack
- PostgreSQL
- Sirius
- Spatial
- SQLite
- SQLSmith
- TPC-DS
- TPC-H
- UI
- Unity Catalog
- Vortex
- VSS
- Quack Remote Protocol
- Guides
- Overview
- Data Viewers
- Database Integration
- File Formats
- Overview
- CSV Import
- CSV Export
- Directly Reading Files
- Directly Reading DuckDB Databases
- Excel Import
- Excel Export
- JSON Import
- JSON Export
- Parquet Import
- Parquet Export
- Querying Parquet Files
- File Access with the file: Protocol
- Meta Queries
- Describe Table
- EXPLAIN: Inspect Query Plans
- EXPLAIN ANALYZE: Profile Queries
- List Tables
- Summarize
- DuckDB Environment
- Network and Cloud Storage
- Overview
- HTTP Parquet Import
- HTTP CSV Import
- S3 Parquet Import
- S3 Parquet Export
- S3 Iceberg Import
- S3 Express One
- GCS Import
- Cloudflare R2 Import
- DuckDB over HTTPS / S3
- Share a Views-Only Database
- Fastly Object Storage Import
- SeaweedFS Import
- Tigris Import
- ODBC
- Performance
- Overview
- Environment
- Import
- Schema
- Indexing
- Join Operations
- File Formats
- How to Tune Workloads
- My Workload Is Slow
- Out-of-Memory Issues
- Benchmarks
- Working with Huge Databases
- Python
- Installation
- Executing SQL
- Jupyter Notebooks
- marimo Notebooks
- SQL on Pandas
- Import from Pandas
- Export to Pandas
- Import from Numpy
- Export to Numpy
- SQL on Arrow
- Import from Arrow
- Export to Arrow
- Relational API on Pandas
- Multiple Python Threads
- Integration with Ibis
- Integration with Polars
- Integration with PyTorch
- Using fsspec Filesystems
- SQL Editors
- Beekeeper Studio SQL Editor
- DBeaver SQL IDE
- DBine SQL Editor
- LibreDB Studio SQL Editor
- VisuaLeaf SQL Editor
- SQL Features
- AsOf Join
- Full-Text Search
- Graph Queries
- query and query_table Functions
- Merge Statement for SCD Type 2
- Timestamp Issues
- Snippets
- Creating Synthetic Data
- Dutch Railway Datasets
- Sharing Macros
- Analyzing a Git Repository
- Importing Duckbox Tables
- Copying an In-Memory Database to a File
- Calculating a Database Checksum
- Troubleshooting
- Glossary of Terms
- Browsing Offline
- Operations Manual
- Overview
- Migrating to DuckDB v2.0
- Backup Extension Repositories
- DuckDB's Footprint
- Installing DuckDB
- Logging
- User Agents
- Securing DuckDB
- Non-Deterministic Behavior
- Limits
- DuckDB Docker Container
- Development
- Overview
- DuckDB Repositories
- Release Cycle
- Metrics
- Profiling
- Building DuckDB
- Overview
- Build Configuration
- Building Extensions
- Statically Linking Extensions
- Android
- OpenHarmony
- Linux
- macOS
- Raspberry Pi
- Windows
- Python
- R
- Troubleshooting
- Unofficial and Unsupported Platforms
- Benchmark Suite
- Testing
- Overview
- sqllogictest Introduction
- Writing Tests
- Unit Tester Configuration
- Debugging
- Result Verification
- Persistent Testing
- Loops
- Multiple Connections
- Catch
- Internal Errors
- Internals
- Sitemap
- Live Demo
New Support for defining window functions in extensions was introduced in DuckDB 2.0. The API is part of the C++ extension API and may still change.
Every aggregate function can be used as a window function, but not every windowed operation is an aggregate. For example, fill interpolates missing values, a reduce(init, lambda) function could apply running updates without a frame and a rank(value) function could compute the “hypothetical rank” of a value. Since DuckDB 2.0, window functions are catalog entries, so extensions can define new ones using the same API that the built-in window functions use.
Registering Window Functions
To define a window function, derive from the WindowFunction class and register it in the extension's load function using the ExtensionLoader:
DuckWeedFunction duckweed;
loader.RegisterFunction(duckweed);
Registered window functions are listed by duckdb_functions() with the function type window and can be looked up with the CLI's .manual dot command.
Binding
The WindowFunction constructor takes the function's name, argument types, return type and callbacks. The optional bind callback receives the arguments along with the types of the window's ORDER BY and the argument ORDER BY clauses, so it can resolve the return type and add bind data. The following methods declare which modifiers the function supports:
| Method | Modifier | Default |
|---|---|---|
SetCanDistinct |
DISTINCT arguments |
false |
SetCanFilter |
FILTER clause |
false |
SetCanOrderBy |
ORDER BY arguments |
true |
SetCanExclude |
EXCLUDE clause |
false |
SetCanIgnoreNulls |
IGNORE NULLS |
true |
Blocking Interface
Window functions are typically evaluated in a blocking fashion: the window operator first partitions and sorts the input into hash groups, then evaluates the function on each hash group. A blocking window function implements the following callbacks:
| Callback | Description |
|---|---|
bounds |
Requests the bounds that the function needs: partition (PARTITION BY), peers (RANGE and GROUPS), valid values (NULL handling) and frame (BETWEEN). |
sharing |
Registers the argument expressions so that their values are shared with other window functions, either as a fully materialized collection or as chunks during the sink and evaluate phases. |
global, local |
Construct the global and thread-local states for a hash group. |
sink, finalize |
Make a first pass over the data to construct acceleration structures such as merge sort trees. This phase is fully parallel. There is no combine step, so updates to the global state must be synchronized, e.g., using locks. |
evaluate |
Computes the result Vector for a chunk of rows using the arguments, the index of the first row within the hash group, the requested bounds and the global and local states. Evaluation is fully parallel, so the chunks are usually but not necessarily consecutive. |
Streaming Interface
Some window functions can also be streamed, i.e., evaluated one chunk at a time without materializing the input. DuckDB only considers streaming if the window has no PARTITION BY, no ORDER BY and no EXCLUDE clause. For aggregates, this covers “running totals” (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW). A window function opts in by implementing the following callbacks:
| Callback | Description |
|---|---|
can_stream |
Decides whether the window expression can be streamed, given the maximum number of rows that the operator can buffer. |
streaming_state |
Constructs the state, which derives from WindowExecutorStreamingState. Its offset field sets the buffering offset, which is how lead and lag are streamed. Streaming is single-threaded, so there is only one state. |
stream |
Computes the result Vector for a chunk of rows using the input, the buffered (delayed) input and the state. |
Example
The loadable extension demo defines duckweed, a “fill down” window function that replaces NULL values with the last non-NULL value and implements both the blocking and the streaming interface. With the demo extension loaded, the following query uses the streaming interface:
SELECT i, m, duckweed(m) OVER () AS d
FROM (
SELECT i, if(i % 5, i, NULL) AS m
FROM range(8) t(i)
);
| i | m | d |
|---|---|---|
| 0 | NULL | NULL |
| 1 | 1 | 1 |
| 2 | 2 | 2 |
| 3 | 3 | 3 |
| 4 | 4 | 4 |
| 5 | NULL | 4 |
| 6 | 6 | 6 |
| 7 | 7 | 7 |
Adding PARTITION BY or ORDER BY to the OVER clause switches to the blocking interface.
For more details, see Richard Wesley's talk “Window Extensions: Defining New Window Functions”.