ChatGPT's Gastrointestinal Tumor Board Tango: A limping dance partner?

Aghamaliyev, Ughur; Karimbayli, Javad; Giessen-Jung, Clemens; Ilmer, Matthias ; Unger, Kristian; Andrade, Dorian ; Hofmann, Felix O. ; Weniger, Maximilian; Angele, Martin K.; Westphalen, Christoph Benedikt; Werner, Jens and Renz, Bernhard W. (2024): ChatGPT's Gastrointestinal Tumor Board Tango: A limping dance partner? In: European Journal of Cancer, Vol. 205, 114100

Abstract

Objectives: This study aimed to assess the consistency and replicability of treatment recommendations provided by ChatGPT 3.5 compared to gastrointestinal tumor cases presented at multidisciplinary tumor boards (MTBs). It also aimed to distinguish between general and case-specific responses and investigated the precision of ChatGPT’s recommendations in replicating exact treatment plans, particularly regarding chemotherapy regimens and follow-up protocols. Material and methods: A retrospective study was carried out on 115 cases of gastrointestinal malignancies, selected from 448 patients reviewed in MTB meetings. A senior resident fed patient data into ChatGPT 3.5 to produce treatment recommendations, which were then evaluated against the tumor board’s decisions by senior oncology fellows.

Results: Among the examined cases, ChatGPT 3.5 provided general information about the malignancy without considering individual patient characteristics in 19% of cases. However, only in 81% of cases, ChatGPT generated responses that were specific to the individual clinical scenarios. In the subset of case-specific responses, 83% of recommendations exhibited overall treatment strategy concordance between ChatGPT and MTB. However, the exact treatment concordance dropped to 65%, notably lower in recommending specific chemotherapy regimens. Cases recommended for surgery showed the highest concordance rates, while those involving chemotherapy recommendations faced challenges in precision.

Conclusions: ChatGPT 3.5 demonstrates potential in aligning conceptual approaches to treatment strategies with MTB guidelines. However, it falls short in accurately duplicating specific treatment plans, especially concerning chemotherapy regimens and ollow-up procedures. Ethical concerns and challenges in achieving exact replication necessitate prudence when considering ChatGPT 3.5 for direct clinical decision-making in MTBs.

• Item Type: Journal article
• Faculties: Medicine > Medical Center of the University of Munich; Medicine > Medical Center of the University of Munich > Clinic for General, Visceral, Vascular and Transplant Surgery; Medicine > Medical Center of the University of Munich > Medical Clinic and Outpatient Clinic III (Oncology); Medicine > Medical Center of the University of Munich > Clinic and Polyclinic for Radiotherapy and Radiooncology
• Subjects: 600 Technology > 610 Medicine and health
• ISSN: 09598049
• Language: English