bei Google ScholarTitelangaben
Hemmer, Patrick ; Thede, Lukas ; Vössing, Michael ; Jakubik, Johannes ; Kühl, Niklas:
Learning to Defer with Limited Expert Predictions.
In: Proceedings of the AAAI Conference on Artificial Intelligence.
Bd. 37
(2023)
Heft 5
.
- S. 6002-6011.
ISSN 2159-5399
DOI: https://doi.org/10.1609/aaai.v37i5.25742
Abstract
Recent research suggests that combining AI models with a human expert can exceed the performance of either alone. The combination of their capabilities is often realized by learning to defer algorithms that enable the AI to learn to decide whether to make a prediction for a particular instance or defer it to the human expert. However, to accurately learn which instances should be deferred to the human expert, a large number of expert predictions that accurately reflect the expert's capabilities are required -- in addition to the ground truth labels needed to train the AI. This requirement shared by many learning to defer algorithms hinders their adoption in scenarios where the responsible expert regularly changes or where acquiring a sufficient number of expert predictions is costly. In this paper, we propose a three-step approach to reduce the number of expert predictions required to train learning to defer algorithms. It encompasses (1) the training of an embedding model with ground truth labels to generate feature representations that serve as a basis for (2) the training of an expertise predictor model to approximate the expert's capabilities. (3) The expertise predictor generates artificial expert predictions for instances not yet labeled by the expert, which are required by the learning to defer algorithms. We evaluate our approach on two public datasets. One with "synthetically" generated human experts and another from the medical domain containing real-world radiologists' predictions. Our experiments show that the approach allows the training of various learning to defer algorithms with a minimal number of human expert predictions. Furthermore, we demonstrate that even a small number of expert predictions per class is sufficient for these algorithms to exceed the performance the AI and the human expert can achieve individually.
Weitere Angaben
| Publikationsform: | Artikel in einer Zeitschrift |
|---|---|
| Begutachteter Beitrag: | Ja |
| Institutionen der Universität: | Fakultäten > Rechts- und Wirtschaftswissenschaftliche Fakultät > Fachgruppe Betriebswirtschaftslehre Fakultäten > Rechts- und Wirtschaftswissenschaftliche Fakultät > Fachgruppe Betriebswirtschaftslehre > Lehrstuhl Wirtschaftsinformatik > Lehrstuhl Wirtschaftsinformatik - Univ.-Prof. Dr.-Ing. Niklas Kühl Fakultäten Fakultäten > Rechts- und Wirtschaftswissenschaftliche Fakultät Fakultäten > Rechts- und Wirtschaftswissenschaftliche Fakultät > Fachgruppe Betriebswirtschaftslehre > Lehrstuhl Wirtschaftsinformatik |
| Titel an der UBT entstanden: | Nein |
| Themengebiete aus DDC: | 000 Informatik,Informationswissenschaft, allgemeine Werke > 004 Informatik 300 Sozialwissenschaften > 330 Wirtschaft |
| Eingestellt am: | 04 Mai 2023 06:42 |
| Letzte Änderung: | 08 Mär 2024 10:37 |
| URI: | https://eref.uni-bayreuth.de/id/eprint/76162 |