Publication
Structuring Annotation Label Spaces by Natural Language Concept Elicitation and Ontology Grounding
Pratik Sitapara; Prathmesh Prakash Doddanawar; Thiago S. Gouvêa; Danniel Sonntag
In: Proceedings of the Joint Workshop on Generative AI and Knowledge Graphs and Knowledge Graphs & Agentic Systems Interplay (GenAIK-NORA 2026), co-located with IJCAI-ECAI 2026. The Joint Workshop on Generative AI and Knowledge Graphs and Knowledge Graphs & Agentic Systems Interplay (GenAIK-NORA-2026), located at IJCAI-ECAI 2026, August 17, Bremen, Germany, CEUR-WS, 2026.
Abstract
Annotation remains a central bottleneck in data-centric AI workflows, particularly in expert-driven domains where labels encode complex and evolving domain knowledge. Existing annotation tools typically rely on flat, task-specific label sets that lack semantic structure, interoperability, and principled integration with established knowledge bases. Conversely, authoritative ontologies provide shared vocabularies and formal relations but are often too rigid and difficult for domain experts to navigate and extend during annotation setup. We present a demonstration system for LLM-assisted label space construction, enabling experts to define, structure, and ground annotation concepts through natural language interaction. The system follows a grounding-first strategy that prioritises alignment with authoritative ontologies and domain-relevant knowledge sources. When experts introduce novel concepts not covered by existing resources, the system generates local OWL-based concept representations with explicit provenance. The resulting session-specific ontology defines a fixed semantic contract for subsequent annotation, ensuring both semantic precision and task-level flexibility. We demonstrate the workflow in a bioacoustic annotation scenario and illustrate how human-in-the-loop, tool-mediated grounding can connect natural language concept elicitation with ontology-based label-space construction.