Navigating the Synthetic Realm: Harnessing Diffusion-Based Models for Laparoscopic Text-to-Image Generation

Title data

Allmendinger, Simeon ; Hemmer, Patrick ; Queisner, Moritz ; Sauer, Igor ; Müller, Leopold ; Jakubik, Johannes ; Vössing, Michael ; Kühl, Niklas:
Navigating the Synthetic Realm: Harnessing Diffusion-Based Models for Laparoscopic Text-to-Image Generation.
In: Shaban-Nejad, Arash ; Michalowski, Martin ; Bianco, Simone (ed.): AI for Health Equity and Fairness : Leveraging AI to Address Social Determinants of Health. - Cham : Springer , 2024 . - pp. 31-46
ISBN 978-3-031-63592-2
DOI: https://doi.org/10.1007/978-3-031-63592-2_4

Project information

Abstract in another language

Recent advances in synthetic imaging open up opportunities for obtaining additional data in the field of surgical imaging. This data can provide reliable supplements supporting surgical applications and decision-making through computer vision. Particularly the field of image-guided surgery, such as laparoscopic and robotic-assisted surgery, benefits strongly from synthetic image datasets and virtual surgical training methods. Our study presents an intuitive approach for generating synthetic laparoscopic images from short text prompts using diffusion-based generative models. We demonstrate the usage of state-of-the-art text-to-image architectures in the context of laparoscopic imaging with regard to the surgical removal of the gallbladder. Results on fidelity and diversity demonstrate that diffusion-based models can acquire knowledge about the style and semantics of image-guided surgery. A validation study with a human assessment survey underlines the realistic nature of our synthetic data, as medical personnel detects actual images in a pool with generated images causing a false-positive rate of 66%. In addition, the investigation of a state-of-the-art machine learning model to recognize surgical actions indicates enhanced results when trained with additional generated images of up to 5.20%. Overall, the achieved image quality contributes to the usage of computer-generated images in surgical applications and enhances its path to maturity.