Exploring Mobile Touch Interaction with Large Language Models

Title data

Zindulka, Tim ; Sekowski, Jannek Maximilian ; Lehmann, Florian ; Buschek, Daniel:
Exploring Mobile Touch Interaction with Large Language Models.
In: Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. - New York, NY, USA : Association for Computing Machinery , 2025 . - 727
ISBN 979-8-4007-1394-1
DOI: https://doi.org/10.1145/3706598.3713554

Project information

Abstract in another language

Interacting with Large Language Models (LLMs) for text editing on mobile devices currently requires users to break out of their writing environment and switch to a conversational AI interface. In this paper, we propose to control the LLM via touch gestures performed directly on the text. We first chart a design space that covers fundamental touch input and text transformations. In this space, we then concretely explore two control mappings: spread-to-generate and pinch-to-shorten, with visual feedback loops. We evaluate this concept in a user study (N=14) that compares three feedback designs: no visualisation, text length indicator, and length + word indicator. The results demonstrate that touch-based control of LLMs is both feasible and user-friendly, with the length + word indicator proving most effective for managing text generation. This work lays the foundation for further research into gesture-based interaction with LLMs on touch devices.