Abstract:

In the rapidly evolving healthcare landscape, Artificial Intelligence (AI) emerges as a transformative innovation poised to significantly enhance patient care. This study evaluates Medily AI and compares its performance with ChatGPT-4 using objective measures. The primary objective is to assess Medily AI's potential to improve the efficiency, accuracy, and accessibility of healthcare delivery in the UK.

Methods:
To evaluate Medily AI’s effectiveness, we analysed its responses to 200 publicly available UKMLA practice questions and six clinical cases sourced from MedPix, a comprehensive medical image database. We benchmarked Medily AI’s outputs against those of practicing clinicians and ChatGPT-4. Additionally, we conducted a survey among healthcare professionals to gauge their perspectives on integrating AI into clinical practice.

Results:
Medily AI demonstrated robust diagnostic accuracy, achieving an average success rate of 89.0% on UKMLA questions, surpassing ChatGPT-4’s performance at 76.3%. In the MedPix cases, Medily AI consistently matched or closely approximated responses from experienced clinicians, confirming its proficiency in generating precise differential diagnoses and treatment plans. Survey results indicated that 95% of healthcare professionals recognise the potential benefits of AI in patient care, with 76% frequently encountering scenarios where AI could enhance clinical outcomes.

Discussion:
These findings underscore AI’s potential to revolutionise healthcare by offering precise, evidence-based diagnostic support. Its consistent performance in clinical scenarios suggests that AI tools like Medily AI can significantly alleviate medical staff workloads, improve patient outcomes, and promote a more efficient healthcare system. Continued research and development are essential to fully exploit AI's potential.

References:
[1] Lai, U.H., et al. (2023). Evaluating the performance of ChatGPT-4 on the United Kingdom Medical Licensing Assessment. Journal of Medical Internet Research, 25(10), 1234-1245. doi:10.2196/37795422.
[2] MedPix®. (2024). National Library of Medicine. Available at: https://medpix.nlm.nih.gov/ (Accessed: 24 April 2024).
[3] Medily AI. (2024). Available at: https://medily.co.uk (Accessed: 24 April 2024).
[4] ChatGPT. (2024). Available at: https://chatgpt.com (Accessed: 24 April 2024).
[5] General Medical Council. (2024). Medical Licensing Assessment. Available at: https://www.gmc-uk.org/education/medical-licensing-assessment (Accessed: 24 April 2024).
[6] Medical Schools Council. (2024). Practice exam for the MS AKT. Available at: https://www.medschools.ac.uk/medical-licensing-assessment/preparing-for-the-ms-akt/practice-exam-for-the-ms-akt (Accessed: 24 April 2024).

Keywords:

Artificial Intelligence, healthcare, diagnostics, patient care, clinical practice, Medily AI, ChatGPT, Medical Exams, Medical Education.