Abstract:
OBJECTIVE To evaluate the performance of five mainstream Chinese large language models (LLMs) in supporting decision-making for the prevention and control of
Candida auris healthcare-associated infection (HAI).
METHODS Based on the Expert Consensus on the Prevention and Control of
Candida auris Health Care-associated Infection, a test set comprising 24 questions across six dimensions was developed and stratified into factual, procedural, and judgment-based tasks. A cross-sectional evaluation was conducted on Qwen, DeepSeek, GLM, Doubao, and ERNIE Bot. Three infection control professionals assessed the overall quality, accuracy, and comprehensiveness of model-generated responses by using an operationalized scoring rubric. The inter-rater reliability was evaluated by the intraclass correlation coefficient (ICC).
RESULTS The overall ICC of the three raters was 0.927, indicating excellent agreement. Qwen achieved the highest total score (360 points), followed by DeepSeek (358 points). All of the models achieved full scores in the basic knowledge and management dimension and showed relatively stable performance in standardized dimensions such as hand hygiene and personal protection, and patient isolation management. The performance losses were mainly observed in screening and monitoring, skin disinfection preparation and patient isolation management, which involved specialized details, application condition and key numerical dimensions. The Friedman test showed a significant difference in the overall performance scores among the five models (
χ2=14.400,
P= 0.006), as well as a significant difference in the accuracy for factual tasks (
P= 0.045).
CONCLUSIONS LLMs exhibit certain potential for decision support in control of the
C. auris infection and show generally high accuracy, powerful information integration, and text generation capabilities, and they may serve as tools for information retrieval, process reminders, and preliminary recommendation generation, but should not replace the professional clinical judgment.