The environment where DuckDB is run has an obvious impact on performance. This page focuses on the effects of the hardware configuration and the operating system used.

Hardware Configuration

CPU

DuckDB works efficiently on both AMD64 (x86_64) and ARM64 (AArch64) CPU architectures.

Memory

Bestpractice Aim for 1-4 GB memory per thread.

Minimum Required Memory

As a rule of thumb, DuckDB requires a minimum of 125 MB of memory per thread. For example, if you use 8 threads, you need at least 1 GB of memory. If you are working in a memory-constrained environment, consider limiting the number of threads, e.g., by issuing:

SET threads = 4;

Memory for Ideal Performance

The amount of memory required for ideal performance depends on several factors, including the data set size and the queries to execute. Maybe surprisingly, the queries have a larger effect on the memory requirement. Workloads containing large joins over many-to-many tables yield large intermediate datasets and thus require more memory for their evaluation to fully fit into the memory. As an approximation, aggregation-heavy workloads require 1-2 GB memory per thread and join-heavy workloads require 3-4 GB memory per thread.

Larger-than-Memory Workloads

DuckDB can process larger-than-memory workloads by spilling to disk. This is possible thanks to out-of-core support for grouping, joining, sorting and windowing operators. Note that larger-than-memory workloads can be processed both in persistent mode and in in-memory mode as DuckDB still spills to disk in both modes.

Local Disk

Disk type. DuckDB's disk-based mode is designed to work best with SSD and NVMe disks. While HDDs are supported, they will result in low performance, especially for write operations.

Disk-based vs. in-memory storage. Counter-intuitively, using a disk-based DuckDB instance can be faster than an in-memory instance due to compression. Read more in the “How to Tune Workloads” page.

File systems. On Linux, DuckDB performs best with the XFS file system but it also performs reasonably well with other file systems such as ext4. On Windows, we recommend using NTFS and avoiding FAT32.

Note that DuckDB databases have built-in checksums, so integrity checks from the file system are not required to prevent data corruption.

Network-Attached Disks

Cloud disks. DuckDB runs well on network-backed cloud disks such as AWS EBS for both read-only and read-write workloads.

Network-attached storage. Network-attached storage can serve DuckDB for read-only workloads. However, it is not recommended to run DuckDB in read-write mode on network-attached storage (NAS). These setups include NFS, network drives such as SMB and Samba. Based on user reports, running read-write workloads on network-attached storage can result in slow and unpredictable performance, as well as spurious errors cased by the underlying file system.

Warning Avoid running DuckDB in read-write mode on network-attached storage.

Bestpractice Fast disks are important if your workload is larger than memory and/or fast data loading is important. Only use network-backed disks if they are reliable (e.g., cloud disks) and guarantee high IO.

Operating System

We recommend using the latest stable version of operating systems: macOS, Windows, and Linux are all well-tested and DuckDB can run on them with high performance.

Linux

DuckDB runs on all mainstream Linux distributions released in the last ≈5 years. If you don't have a particular preference, we recommended using Ubuntu Linux LTS due to its stability and the fact that most of DuckDB’s Linux test suite jobs run on Ubuntu workers.

glibc vs. musl libc

DuckDB can be built with both glibc (default) and musl libc (see the build guide). However, note that DuckDB binaries built with musl libc have lower performance. In practice, this can lead to a slowdown of more than 5× on compute-intensive workloads. Therefore, it's recommended to use a Linux distribution with glibc for performance-oriented workloads when running DuckDB.

Memory Allocator

If you have a many-core CPU running on a system where DuckDB ships with jemalloc as the default memory allocator, consider enabling the allocator's background threads.