LLM-Based Integration of Contextual Knowledge in Natural Language Commands for Enhanced Robot Programming

Titelangaben

Sucker, Sascha ; Khaleghian, Sahar ; Henrich, Dominik:
LLM-Based Integration of Contextual Knowledge in Natural Language Commands for Enhanced Robot Programming.
2026
Veranstaltung: 35th International Conference on Robotics in Alpe-Adria-Danube Region (RAAD 2026) , 17.06.-19.06.2026 , Bratislava, Slovakia.
(Veranstaltungsbeitrag: Kongress/Konferenz/Symposium/Tagung , Vortrag mit Paper )
DOI: https://doi.org/10.1007/978-3-032-29127-1_9

Angaben zu Projekten

Abstract

Natural language is an intuitive interface for non-expert robot programming, but remains unreliable due to speech recognition errors and inherent linguistic ambiguities. As users issue commands based on their expectations of the robot and its environment, they may contradict the programming system’s constraints. We bridge this contradiction by enriching the transcriptions with structured context knowledge about the robot, task domain, and environment. Rather than generating robot programs directly, a large language model refines transcriptions, which are interpreted by a programming system. Thereby, the programming system preserves predictability and formal correctness while the LLM expands the range of admissible inputs. We evaluate the effect of different amounts of context on programming correctness using spoken commands collected in a user study (N=22). Our results show an increase in programming correctness for contextual enrichment (from 8.5% to 49.5%), suggesting that command enrichment can enhance programming systems.