bei Google ScholarTitelangaben
Neuberger, Julian ; Ackermann, Lars ; van der Aa, Han ; Jablonski, Stefan:
A Universal Prompting Strategy for Extracting Process Model Information from Natural Language Text Using Large Language Models.
In: Maass, Wolfgang ; Han, Hyoil ; Yasar, Hasan ; Multari, Nick
(Hrsg.):
Conceptual Modeling : Proceedings. -
Cham, Switzerland
: Springer Nature
,
2025
. - S. 38-55
. - (Lecture Notes in Computer Science
; 15238
)
ISBN 978-3-031-75872-0
DOI: https://doi.org/10.1007/978-3-031-75872-0_3
Abstract
Over the past decade, extensive research efforts have been dedicated to the extraction of information from textual process descriptions. Despite the remarkable progress witnessed in natural language processing (NLP), information extraction within the Business Process Management domain remains predominantly reliant on rule-based systems and machine learning methodologies. Data scarcity has so far prevented the successful application of deep learning techniques. However, the rapid progress in generative large language models (LLMs) makes it possible to solve many NLP tasks with very high quality without the need for extensive data. Therefore, we systematically investigate the potential of LLMs for extracting information from textual process descriptions, targeting the detection of process elements such as activities and actors, and relations between them. Based on a novel prompting strategy, we show that LLMs are able to outperform state-of-the-art machine learning approaches with absolute performance improvements of up to 8% $$F_1$$F1score across three different datasets. We evaluate our prompting strategy on eight different LLMs, showing it is universally applicable, while also analyzing the impact of certain prompt parts on extraction quality. The number of example texts, the specificity of definitions, and the rigour of format instructions are identified as key for improving the accuracy of extracted information. Our code, prompts, and data are publicly available at https://github.com/JulianNeuberger/llm-process-generation/tree/er2024.
Weitere Angaben
| Publikationsform: | Aufsatz in einem Buch |
|---|---|
| Begutachteter Beitrag: | Ja |
| Institutionen der Universität: | Fakultäten > Fakultät für Mathematik, Physik und Informatik > Institut für Informatik > Lehrstuhl Angewandte Informatik IV > Lehrstuhl Angewandte Informatik IV - Univ.-Prof. Dr.-Ing. Stefan Jablonski |
| Titel an der UBT entstanden: | Ja |
| Themengebiete aus DDC: | 000 Informatik,Informationswissenschaft, allgemeine Werke > 004 Informatik |
| Eingestellt am: | 30 Okt 2024 11:20 |
| Letzte Änderung: | 30 Okt 2024 11:20 |
| URI: | https://eref.uni-bayreuth.de/id/eprint/90902 |