DuckDB GCS Extension
Maintainer(s):
northpolesec
Installing and Loading
INSTALL gcs FROM community;
LOAD gcs;
Example
-- Add auth credentials using Application Default Creds
D CREATE SECRET secret (TYPE gcp, PROVIDER credential_chain);
-- Read a file from GCS
D SELECT * FROM read_text('gcss://rah-public-gcs-testing/quack.txt');
┌─────────────────────────────────────────┬─────────┬───────┬──────────────────────────┐
│ filename │ content │ size │ last_modified │
│ varchar │ varchar │ int64 │ timestamp with time zone │
├─────────────────────────────────────────┼─────────┼───────┼──────────────────────────┤
│ gcss://rah-public-gcs-testing/quack.txt │ 🦆 │ 4 │ 2025-09-23 16:20:03-04 │
└─────────────────────────────────────────┴─────────┴───────┴──────────────────────────┘
Added Settings
| name | description | input_type | scope | aliases |
|---|---|---|---|---|
| gcs_enable_metadata_cache | Enable caching of object metadata (size, modification time) to reduce API calls. Set to false to disable caching for debugging or when files change frequently. | BOOLEAN | GLOBAL | [] |
| gcs_list_cache_ttl | Time-to-live in seconds for cached object listing results (used in glob operations). Default is 60 seconds. Increase for stable directories, decrease if objects are added/removed frequently. | INTEGER | GLOBAL | [] |
| gcs_max_list_cache_entries | Maximum number of list cache entries to prevent unbounded memory growth. Default is 1000. When limit is reached, least recently used entries are evicted. | UBIGINT | GLOBAL | [] |
| gcs_max_metadata_cache_entries | Maximum number of metadata cache entries to prevent unbounded memory growth. Default is 10000. When limit is reached, least recently used entries are evicted. | UBIGINT | GLOBAL | [] |
| gcs_metadata_cache_ttl | Time-to-live in seconds for cached object metadata. Default is 300 seconds (5 minutes). Increase for stable files, decrease if files change frequently. | INTEGER | GLOBAL | [] |
| gcs_transfer_concurrency | Number of concurrent worker threads to use when reading. Default is 5. | INTEGER | GLOBAL | [] |