SFBorg: a frictionless biodiversity data exchange
Dmitry Mozzherin · Semantic Interoperability and Standards Infrastructure
The short versionUsing an SQLite schema as the exchange format lets biodiversity datasets be used instantly by applications without import.
What this was about
Dmitry Mozzherin presents SFBorg and its Species File Group Archive (SFGA) format, an SQLite database schema used as an "active" exchange format that applications can use directly without importing, by analogy with MP3 files. SQLite is a single, fast, compressible file format with a stable format supported until 2050 and accepted by the Library of Congress for archiving. The ecosystem includes a shared library, the SF conversion tool (e.g. CSV, Darwin Core, ColDP via SFGA), a Harvester for idiosyncratic datasets, GNdb for building name-resolution databases, and Hive, a taxonomic viewer/editor/validator.
Why it matters. Text-based exchange formats require database import skills; an active single-file format could enable many zero-effort applications for any dataset.
In the room
- Darwin Core Archives, ColDP and Darwin Core Data Packages are inert text files that must be imported.
- SFGA is a semantically versioned SQLite schema; version 1 will guarantee backward compatibility.
- SQLite: single file, fast, scalable to terabytes, standard SQL, format stable for 30 years and supported until at least 2050.
- SQLite is a Library of Congress recommended archival format alongside XML and JSON.
- Used in production at Species File Group for harvesting, conversion (Darwin Core to ColDP and back), validation and aggregation (e.g. GN Verifier).
- Tools: Species File Library, SF converter, Harvester, GNdb, Hive.
- Ideas: instant websites, curation apps and AI interfaces for any SFGA dataset.
Notable moments
Transcript
Automatically generated captions can contain mistakes, especially in names and technical terms. Times are relative to the room recording.
Read the transcript ↓
Related talks.
Machine-Ready Botanical Data: Mapping Integrated Collections and Traits with Darwin Core Data Package
Darwin Core Data Package can serve as a semantic integration layer for a botanical garden's diverse collections and trait data, and MBG's draft mapping is offered for community refinement.
Architecting the National Biodiversity Information System: Standards-Driven Workflows for South Africa’s Botanical and Zoological Data
SANBI sustains NBIS through a statutory mandate plus a roadmap of diversified funding, partnerships and demonstrated return on investment.
Enabling machine-readable access to biodiversity data through federated semantic querying
Distribute the data, not the infrastructure: cloud Parquet plus OBDA query rewriting gives a virtual Darwin Core knowledge graph without duplicating data into RDF.