Read, write, validate and edit MARC 21 bibliographic data (ISO 2709, MARCXML, MARC-in-JSON, breaker, Aleph; UTF-8 and MARC-8)
Maintainer(s):
mgbilby
Installing and Loading
INSTALL marc21 FROM community;
LOAD marc21;
Example
-- 1. Read a binary MARC dump (gzip and globs work; MARC-8 and UTF-8
-- are detected per record and decoded to NFC Unicode)
SELECT control_number,
marc_subfield(fields, '245', 'a') AS title,
marc_subfields(fields, '650', 'a') AS subjects
FROM read_marc('catalogue/*.mrc.gz');
-- 2. One row per subfield, for frequency and completeness work
SELECT tag, code, count(*) FROM read_marc_subfields('dump.mrc')
GROUP BY ALL ORDER BY 3 DESC;
-- 3. MARCXML, MARC-in-JSON (incl. FOLIO SRS shape), breaker, Aleph
SELECT * FROM read_marcxml('export.xml');
SELECT * FROM read_marcjson('records.ndjson');
-- 4. Address by MARCspec; validate structurally or against Avram
SELECT marc_spec(leader, fields, '008/35-37') FROM read_marc('dump.mrc');
SELECT marc_validate(leader, fields) FROM read_marc('dump.mrc');
-- 5. Batch-edit and write back — full encoding crosswalk
COPY (SELECT marc_set_subfield(fields, '040', 'a', 'XX-XxUND') AS fields,
leader
FROM read_marc('in.mrc'))
TO 'out.mrc' (FORMAT marc, ENCODING 'marc8');
-- 6. Or write MARCXML / breaker / MARC-in-JSON
COPY (SELECT * FROM read_marc('in.mrc')) TO 'out.xml' (FORMAT marcxml);
About marc21
A cleanly licensed MARC 21 toolkit for DuckDB, aimed at cataloging and metadata librarians: the parser, MARC-8 tables and writer are implemented from ISO 2709 and the Library of Congress specifications only, not from previous software, and verified by differential fuzzing and round-trip testing.
Read
read_marc (nested LIST(STRUCT) per record), read_marc_subfields
(one row per subfield), read_marc_raw (original bytes + error column),
read_marcxml, read_marc_breaker, read_marcjson (community shape,
NDJSON and FOLIO SRS envelopes), read_alephseq. Glob patterns, gzip,
and DuckDB FileSystem protocols (httpfs URLs — including SRU and OAI-PMH
responses, whose envelopes are handled). read_z3950 retrieves records
live from Z39.50 servers (Library of Congress, OCLC, most ILSes) with a
built-in dependency-free client. Scans parallelise across and within
files.
Write
COPY ... TO (FORMAT marc | marcxml | mrk | marcjson). ISO 2709 output
in UTF-8 or MARC-8 (full LC code tables: ANSEL, Cyrillic, Greek, Hebrew,
Arabic, EACC with escape designations; NCR fallback), round-trip tested.
Query, validate, edit
MARCspec addressing (marc_spec), structural and Avram-schema validation
(marc_validate, marc_validate_avram), format detection, record
editing (marc_set_subfield, marc_remove_fields, marc_replace_values,
…), merge with field protection (marc_merge), record diff
(marc_diff), identifier normalisation (marc_isbn13, marc_issn,
marc_lccn, marc_oclc), NACO normalisation and match keys for
deduplication, and a FOLIO Source Record Storage bridge
(marc_parse_json) for in-place analytics on an ATTACHed FOLIO database.
Added Functions
| function_name | function_type | description | comment | examples |
|---|---|---|---|---|
| marc_008_books | macro | NULL | NULL | |
| marc_008_parts | macro | NULL | NULL | |
| marc_008_struct | macro | NULL | NULL | |
| marc_260c_copyright_year | macro | NULL | NULL | |
| marc_264_copyright | macro | NULL | NULL | |
| marc_264_from_260 | macro | NULL | NULL | |
| marc_264_publication | macro | NULL | NULL | |
| marc_86x_zip | macro | NULL | NULL | |
| marc_add_field | scalar | NULL | NULL | |
| marc_add_local | macro | NULL | NULL | |
| marc_alma_bib_body | macro | NULL | NULL | |
| marc_alma_headers | macro | NULL | NULL | |
| marc_alma_put_sql | macro | NULL | NULL | |
| marc_alma_put_url | macro | NULL | NULL | |
| marc_alma_sql_quote | macro | NULL | NULL | |
| marc_alma_update_bib | table_macro | NULL | NULL | |
| marc_alma_xml_escape | macro | NULL | NULL | |
| marc_authority_refs | macro | NULL | NULL | |
| marc_bf_chomp | macro | NULL | NULL | |
| marc_bf_chompd | macro | NULL | NULL | |
| marc_bf_classes | macro | NULL | NULL | |
| marc_bf_contrib | macro | NULL | NULL | |
| marc_bf_contribs | macro | NULL | NULL | |
| marc_bf_date | macro | NULL | NULL | |
| marc_bf_id | macro | NULL | NULL | |
| marc_bf_ident | macro | NULL | NULL | |
| marc_bf_idents | macro | NULL | NULL | |
| marc_bf_instance | macro | NULL | NULL | |
| marc_bf_lang | macro | NULL | NULL | |
| marc_bf_obj | macro | NULL | NULL | |
| marc_bf_provision | macro | NULL | NULL | |
| marc_bf_pubfield | macro | NULL | NULL | |
| marc_bf_sf | macro | NULL | NULL | |
| marc_bf_sfjoin | macro | NULL | NULL | |
| marc_bf_sfvals | macro | NULL | NULL | |
| marc_bf_str | macro | NULL | NULL | |
| marc_bf_subject | macro | NULL | NULL | |
| marc_bf_subjects | macro | NULL | NULL | |
| marc_bf_title | macro | NULL | NULL | |
| marc_bf_vocab | macro | NULL | NULL | |
| marc_bf_work | macro | NULL | NULL | |
| marc_bf_worktype | macro | NULL | NULL | |
| marc_bib_level | macro | NULL | NULL | |
| marc_bibframe_jsonld | macro | NULL | NULL | |
| marc_build_field | scalar | NULL | NULL | |
| marc_change_case | scalar | NULL | NULL | |
| marc_check_urls | table_macro | NULL | NULL | |
| marc_clear_linked_uris | scalar | NULL | NULL | |
| marc_cluster_headings | table_macro | NULL | NULL | |
| marc_control_field | macro | NULL | NULL | |
| marc_control_pos | macro | NULL | NULL | |
| marc_copy_field | scalar | NULL | NULL | |
| marc_cutter_valid | scalar | NULL | NULL | |
| marc_ddc_sortkey | scalar | NULL | NULL | |
| marc_dedupe_candidates | table_macro | NULL | NULL | |
| marc_dedupe_key | macro | NULL | NULL | |
| marc_delimited_assemble | macro | NULL | NULL | |
| marc_delimited_build | macro | NULL | NULL | |
| marc_delimited_clean | macro | NULL | NULL | |
| marc_delimited_preset_books | macro | NULL | NULL | |
| marc_delimited_preset_eresources | macro | NULL | NULL | |
| marc_delimited_preset_serials | macro | NULL | NULL | |
| marc_delimited_tagsort | macro | NULL | NULL | |
| marc_diff | scalar | NULL | NULL | |
| marc_dublin_core | macro | NULL | NULL | |
| marc_encoding_scheme | macro | NULL | NULL | |
| marc_expand_863 | macro | NULL | NULL | |
| marc_fieldbreak | macro | NULL | NULL | |
| marc_fields | macro | NULL | NULL | |
| marc_fingerprint | scalar | NULL | NULL | |
| marc_folio_check | macro | NULL | NULL | |
| marc_format | macro | NULL | NULL | |
| marc_from_delimited | macro | NULL | NULL | |
| marc_from_delimited_url | macro | NULL | NULL | |
| marc_from_unimarc_label | macro | NULL | NULL | |
| marc_generate_33x | scalar | NULL | NULL | |
| marc_heading_flips | macro | NULL | NULL | |
| marc_heading_join | macro | NULL | NULL | |
| marc_headings | macro | NULL | NULL | |
| marc_headings_naco | macro | NULL | NULL | |
| marc_holdings_pairs | macro | NULL | NULL | |
| marc_idloc_candidates | macro | NULL | NULL | |
| marc_idloc_label_url | macro | NULL | NULL | |
| marc_idloc_suggest_url | macro | NULL | NULL | |
| marc_ids_fast | macro | NULL | NULL | |
| marc_ids_lc | macro | NULL | NULL | |
| marc_ids_viaf | macro | NULL | NULL | |
| marc_instance | macro | NULL | NULL | |
| marc_isbd | macro | NULL | NULL | |
| marc_isbd_area1 | macro | NULL | NULL | |
| marc_isbd_area2 | macro | NULL | NULL | |
| marc_isbd_area4 | macro | NULL | NULL | |
| marc_isbd_area5 | macro | NULL | NULL | |
| marc_isbd_area6 | macro | NULL | NULL | |
| marc_isbd_area8 | macro | NULL | NULL | |
| marc_isbd_nn | macro | NULL | NULL | |
| marc_isbn13 | scalar | NULL | NULL | |
| marc_issn | scalar | NULL | NULL | |
| marc_jsonld | macro | NULL | NULL | |
| marc_jsonld_arr | macro | NULL | NULL | |
| marc_jsonld_body | macro | NULL | NULL | |
| marc_jsonld_str | macro | NULL | NULL | |
| marc_jsonld_type | macro | NULL | NULL | |
| marc_kbart_856 | macro | NULL | NULL | |
| marc_kbart_to_marc | macro | NULL | NULL | |
| marc_lcc_parse | scalar | NULL | NULL | |
| marc_lcc_sortkey | scalar | NULL | NULL | |
| marc_lccn | scalar | NULL | NULL | |
| marc_leader_pos | macro | NULL | NULL | |
| marc_leader_struct | macro | NULL | NULL | |
| marc_matchkey | scalar | NULL | NULL | |
| marc_merge | scalar | NULL | NULL | |
| marc_mods_origininfo | macro | NULL | NULL | |
| marc_mods_xml | macro | NULL | NULL | |
| marc_move_field | scalar | NULL | NULL | |
| marc_naco | scalar | NULL | NULL | |
| marc_new_record | scalar | NULL | NULL | |
| marc_ngram_fingerprint | scalar | NULL | NULL | |
| marc_oai_extract_token | macro | NULL | NULL | |
| marc_oai_page_url | macro | NULL | NULL | |
| marc_oai_token | table_macro | NULL | NULL | |
| marc_oai_url | macro | NULL | NULL | |
| marc_oclc | scalar | NULL | NULL | |
| marc_parse_json | scalar | NULL | NULL | |
| marc_rank | scalar | NULL | NULL | |
| marc_rda_check | macro | NULL | NULL | |
| marc_rda_expand | scalar | NULL | NULL | |
| marc_read_kbart | table_macro | NULL | NULL | |
| marc_readalma | table_macro | NULL | NULL | |
| marc_readalma_bibs | table_macro | NULL | NULL | |
| marc_readalma_sru | table_macro | NULL | NULL | |
| marc_readfields | table_macro | NULL | NULL | |
| marc_readfieldsbreaker | table_macro | NULL | NULL | |
| marc_readfieldsxml | table_macro | NULL | NULL | |
| marc_readfolio_record | table_macro | NULL | NULL | |
| marc_readfolio_source_records | table_macro | NULL | NULL | |
| marc_readfolio_srs | table_macro | NULL | NULL | |
| marc_readkoha | table_macro | NULL | NULL | |
| marc_readkoha_opac | table_macro | NULL | NULL | |
| marc_readkoha_public | table_macro | NULL | NULL | |
| marc_readnestedbreaker | table_macro | NULL | NULL | |
| marc_readnestedxml | table_macro | NULL | NULL | |
| marc_readoai | table_macro | NULL | NULL | |
| marc_readoai_page | table_macro | NULL | NULL | |
| marc_readsru | table_macro | NULL | NULL | |
| marc_readworldshare | table_macro | NULL | NULL | |
| marc_reconcile_headings | table_macro | NULL | NULL | |
| marc_record_type | macro | NULL | NULL | |
| marc_recordbreak | macro | NULL | NULL | |
| marc_remove_fields | scalar | NULL | NULL | |
| marc_remove_fields_where | scalar | NULL | NULL | |
| marc_remove_subfield | scalar | NULL | NULL | |
| marc_rename_subfield | scalar | NULL | NULL | |
| marc_replace_all | scalar | NULL | NULL | |
| marc_replace_values | scalar | NULL | NULL | |
| marc_report_completeness | table_macro | NULL | NULL | |
| marc_report_errors | table_macro | NULL | NULL | |
| marc_report_subfields | table_macro | NULL | NULL | |
| marc_report_tags | table_macro | NULL | NULL | |
| marc_set_indicators | scalar | NULL | NULL | |
| marc_set_linked_uri | scalar | NULL | NULL | |
| marc_set_subfield | scalar | NULL | NULL | |
| marc_sf01 | macro | NULL | NULL | |
| marc_sort_fields | scalar | NULL | NULL | |
| marc_spec | scalar | NULL | NULL | |
| marc_stamp | macro | NULL | NULL | |
| marc_subfield | macro | NULL | NULL | |
| marc_subfields | macro | NULL | NULL | |
| marc_summary | table_macro | NULL | NULL | |
| marc_summary_fields | table_macro | NULL | NULL | |
| marc_summary_subfields | table_macro | NULL | NULL | |
| marc_swap_fields | scalar | NULL | NULL | |
| marc_trim_punct | macro | NULL | NULL | |
| marc_type_of_resource | macro | NULL | NULL | |
| marc_unimarc_label | macro | NULL | NULL | |
| marc_uris | macro | NULL | NULL | |
| marc_urls_856 | macro | NULL | NULL | |
| marc_validate | macro | NULL | NULL | |
| marc_validate_auth | macro | NULL | NULL | |
| marc_validate_avram | scalar | NULL | NULL | |
| marc_validate_bib | macro | NULL | NULL | |
| marc_validate_format | macro | NULL | NULL | |
| marc_validate_holdings | macro | NULL | NULL | |
| marc_viaf_candidates | macro | NULL | NULL | |
| marc_viaf_search_url | macro | NULL | NULL | |
| marc_wikidata_candidates | macro | NULL | NULL | |
| marc_wikidata_search_url | macro | NULL | NULL | |
| marc_xml_escape | macro | NULL | NULL | |
| marc_xslt_command | macro | NULL | NULL | |
| marc_xslt_functions | table | NULL | NULL | |
| marc_xslt_library | scalar | NULL | NULL | |
| marc_xslt_sheet_file | macro | NULL | NULL | |
| marc_xslt_shell_arg | macro | NULL | NULL | |
| marc_xslt_stylesheet | scalar | NULL | NULL | |
| read_alephseq | table | NULL | NULL | |
| read_marc | table | NULL | NULL | |
| read_marc_breaker | table | NULL | NULL | |
| read_marc_raw | table | NULL | NULL | |
| read_marc_subfields | table | NULL | NULL | |
| read_marcjson | table | NULL | NULL | |
| read_marcxml | table | NULL | NULL | |
| read_microlif | table | NULL | NULL | |
| read_z3950 | table | NULL | NULL |
Overloaded Functions
This extension does not add any function overloads.
Added Types
This extension does not add any types.
Added Settings
This extension does not add any settings.