- Installation
- Guides
- Overview
- Data Import & Export
- CSV Import
- CSV Export
- Parquet Import
- Parquet Export
- Query Parquet
- HTTP Parquet Import
- S3 Parquet Import
- S3 Parquet Export
- JSON Import
- JSON Export
- SQLite Import
- Postgres Import
- Meta Queries
- Python
- Install
- Execute SQL
- Jupyter Notebooks
- SQL on Pandas
- Import From Pandas
- Export To Pandas
- SQL on Arrow
- Import From Arrow
- Export To Arrow
- Relational API on Pandas
- Multiple Python Threads
- DuckDB with Ibis
- DuckDB with Fugue
- DuckDB with Polars
- DuckDB with Vaex
- DuckDB with DataFusion
- DuckDB with fsspec filesystems
- SQL Editors
- Data Viewers
- Documentation
- Connect
- Data Import
- Overview
- Insert Statements
- CSV Files
- Parquet Files
- JSON Files
- Multiple Files
- Partitioning
- Appender
- Client APIs
- Overview
- Python
- R
- Java
- Julia
- C
- Overview
- Startup
- Configure
- Query
- Data Chunks
- Values
- Types
- Prepared Statements
- Appender
- Table Functions
- Replacement Scans
- API Reference
- C++
- Node.js
- Wasm
- ODBC
- CLI
- SQL
- Introduction
- Statements
- Overview
- Select
- Insert
- Delete
- Update
- Create Schema
- Create Table
- Create View
- Create Sequence
- Create Macro
- Drop
- Alter Table
- Copy
- Export
- Attach
- Query Syntax
- SELECT
- FROM & JOIN
- WHERE
- GROUP BY
- GROUPING SETS
- HAVING
- ORDER BY
- LIMIT
- SAMPLE
- UNNEST
- WITH
- WINDOW
- QUALIFY
- VALUES
- FILTER
- Set Operations
- Data Types
- Overview
- NULL Values
- Boolean
- Enum
- Numeric
- Text
- Date
- Timestamp
- Interval
- Blob
- Bitstring
- List
- Struct
- Map
- Union
- Expressions
- Functions
- Overview
- Enum Functions
- Numeric Functions
- Text Functions
- Pattern Matching
- Date Functions
- Timestamp Functions
- Timestamp With Time Zone Functions
- Time Functions
- Interval Functions
- Date Formats
- Date Parts
- Blob Functions
- Bitstring Functions
- Nested Functions
- Utility Functions
- Indexes
- Aggregates
- Window Functions
- Samples
- Information Schema
- Metadata Functions
- Configuration
- Pragmas
- Extensions
- Development
- Testing
- Storage Versions & Format
- Profiling
- Sitemap
- Why DuckDB
- FAQ
- Code of Conduct
- Live Demo
Parquet Metadata
The parquet_metadata
function can be used to query the metadata contained within a Parquet file, which reveals various internal details of the Parquet file such as the statistics of the different columns. This can be useful for figuring out what kind of skipping is possible in Parquet files, or even to obtain a quick overview of what the different columns contain.
SELECT * FROM parquet_metadata('test.parquet');
Below is a table of the columns returned by parquet_metadata
.
Field | Type |
---|---|
file_name | VARCHAR |
row_group_id | BIGINT |
row_group_num_rows | BIGINT |
row_group_num_columns | BIGINT |
row_group_bytes | BIGINT |
column_id | BIGINT |
file_offset | BIGINT |
num_values | BIGINT |
path_in_schema | VARCHAR |
type | VARCHAR |
stats_min | VARCHAR |
stats_max | VARCHAR |
stats_null_count | BIGINT |
stats_distinct_count | BIGINT |
stats_min_value | VARCHAR |
stats_max_value | VARCHAR |
compression | VARCHAR |
encodings | VARCHAR |
index_page_offset | BIGINT |
dictionary_page_offset | BIGINT |
data_page_offset | BIGINT |
total_compressed_size | BIGINT |
total_uncompressed_size | BIGINT |
Parquet Schema
The parquet_schema
function can be used to query the internal schema contained within a Parquet file. Note that this is the schema as it is contained within the metadata of the Parquet file. If you want to figure out the column names and types contained within a Parquet file it is easier to use DESCRIBE
.
-- fetch the column names and column types
DESCRIBE SELECT * FROM 'test.parquet';
-- fetch the internal schema of a parquet file
SELECT * FROM parquet_schema('test.parquet');
Below is a table of the columns returned by parquet_schema
.
Field | Type |
---|---|
file_name | VARCHAR |
name | VARCHAR |
type | VARCHAR |
type_length | VARCHAR |
repetition_type | VARCHAR |
num_children | BIGINT |
converted_type | VARCHAR |
scale | BIGINT |
precision | BIGINT |
field_id | BIGINT |
logical_type | VARCHAR |