LT17 closing Q&A: AI hallucinations, validation and lightning-talk format

What this was about

After the lightning talks the moderators opened questions. Besides questions to individual speakers (captured with those talks), an audience member asked whether speakers found hallucinations when reviewing AI outputs. The answer (apparently from a collaborator of Jenna Kline) was that non-generative ML tools don't hallucinate but misclassify, and accuracy should be evaluated at the ecological task level because of distribution shift; Jack Hollister acknowledged validating vast LLM outputs is an unsolved problem; a relayed comment argued ML behaviour annotation beats humans. The session closed with feedback that Q&A in lightning sessions needs better support.

Why it matters. Highlights evaluation practices for AI in biodiversity and practical lessons for running lightning sessions.

In the room

• Question to all: have you found hallucinations, and does not finding any mean none exist or they were undetected?
• Answer: non-generative ML has no hallucinations but has expected misclassifications; evaluate accuracy at the ecological task level because ML accuracy is misleading once deployed under distribution shift.
• Jack Hollister: LLM outputs are too large to validate fully; how to find small hallucinations is an open question.
• Relayed from Jenna Kline: ML-based behaviour classification can be better than human analysis; latent-space approaches (Max Planck Institute of Animal Behavior) address unnamed behaviours.
• Audience feedback: the one-minute format and app made asking questions hard; moderators suggested better aids listing talks at the end.

Notable moments