From Sensor to Species: AI-Assisted Segmentation, Deep Learning Workflows, and Persistent Biodiversity Records from Earth Observation Systems

What this was about

Kit Lewers argued that remote sensing has a longitudinal biodiversity problem: repeat airborne and satellite acquisitions are processed as separate images, so the same tree can appear in many datasets without a connected history. GBIF would not link such records a year apart. In a proof of concept at NEON's SJER oak savanna, they used off-the-shelf DeepForest crown delineation on RGB imagery, IoU matching across 2023 and 2024, and species candidates constrained by NEON vegetation records from GBIF. The pipeline output Darwin Core occurrences with organismID, identifiedBy 'AI' and basisOfRecord MachineObservation. The weak results (rocks and shadows detected as crowns, a generic cloud model giving one species) were a 'useful failure' showing that model provenance and uncertainty at every step must travel with records. Next steps use NASA G-LiHT's 15-year multi-sensor archive.

Why it matters. Remote sensing could add millions of tree-level records to biodiversity infrastructures, and linking detections of the same individual over time is needed to capture growth, disturbance and loss.

In the room

• Most remote-sensing workflows treat each acquisition separately, so one organism can appear in 10 datasets without an accumulated history.
• GBIF fuzzy matching would probably not link observations of the same organism made a year apart (per conversation with Tim Robertson).
• Test site NEON SJER (San Joaquin Experimental Range), an oak savanna deliberately outside DeepForest's typical use case.
• Pipeline: high-resolution RGB → DeepForest crown delineation → candidate species constrained with NEON field records pulled via pygbif → provenance-rich occurrence export.
• Detected crowns went from 222 in 2023 to 232 in 2024. IoU matching found overlaps but also persistent false detections of rocks and shadows.
• A generic off-the-shelf cloud model identified all trees as one type; biodiversity constraints helped but also hurt a general model.
• Output Darwin Core occurrences used occurrenceID, organismID, scientificName, identificationRemarks, identifiedBy = AI and basisOfRecord = MachineObservation.

Notable moments