A generative AI tool that supports GPs during consultations improves the quality of clinical decision-making without compromising patient safety. This is the finding of a large-scale randomised practice-based trial. Although the technology led to better diagnostic assessments and treatment plans, no demonstrable improvement in patients’ health outcomes was observed in the short term.
According to the researchers, this is one of the first randomised clinical trials to examine not only the performance of AI or healthcare providers in a simulated environment, but also the actual impact at patient level in everyday practice. The study was conducted by researchers from the University of Birmingham and the NIHR Biomedical Research Centre: Birmingham. In total, over 9,600 patients treated at sixteen primary care clinics in Kenya took part.
AI advises, doctor decides
During the study, healthcare providers were randomly assigned to two groups. One group worked with an electronic health record into which an AI support system had been integrated, whilst the other group used the standard system without AI. The AI application, AI Consult, is based on a large language model and analysed the information entered by the healthcare provider into the patient record during the consultation. Based on this, the system generated diagnostic suggestions and treatment recommendations that were in line with Kenya’s national clinical guidelines. Potential areas for attention were displayed using a simple colour-coded system with green, yellow and red alerts.
The AI operated entirely in the background. Patients did not see the recommendations, and the healthcare provider retained full responsibility for all decisions regarding diagnosis, treatment and any referrals. Following the AI’s recommendations was not mandatory. According to lead researcher Bilal Mateen, honorary professor of machine learning for health at the University of Birmingham and Chief AI Officer at PATH, one question was central: “This is one of the first studies to rigorously ask the hardest question about AI in healthcare: whether it actually improves outcomes for patients.”
Better decision-making
The researchers found no statistically significant difference in the number of treatments that failed within fourteen days. In the AI group, this percentage was 2.2 per cent, compared with 2.0 per cent in the control group. The number of hospital admissions and deaths was also comparable, meaning no evidence was found that the use of AI poses risks to patients.
However, an independent panel of experienced doctors assessed the quality of the clinical documentation and treatment plans in the AI group as clearly better. Furthermore, the costs of antibiotics were found to be lower, despite the fact that the total number of antibiotic prescriptions differed only slightly between the two groups. According to the researchers, healthcare providers using AI were more likely to make cost-effective choices when prescribing. Patients’ experiences also remained unchanged. Satisfaction with care was comparable in both groups, suggesting that the use of AI did not negatively affect the interaction between patient and healthcare provider.
Basis for further research
According to the researchers, the results show that generative AI can be safely integrated into existing clinical workflows without compromising healthcare providers’ autonomy or patients’ trust. At the same time, they emphasise that it is much more difficult to see improvements in clinical decision-making directly reflected in health outcomes, particularly within primary care where serious complications are relatively rare. Co-author Alastair Denniston points out that many patients in general practice have conditions that resolve spontaneously or require only limited medical intervention. Consequently, very large studies, potentially involving more than 100,000 participants, are needed to reliably demonstrate small differences in patient outcomes.
The researchers emphasise that, although the study was conducted in Kenya, the results are also relevant to other healthcare systems. However, further research is needed to determine to what extent the findings can be applied to countries where the quality of primary care is already high. According to the researchers, the results primarily offer a realistic picture of what generative AI can currently contribute to daily clinical practice and where future investments and follow-up research should be focused.
Mental healthcare
Last year, researchers at the University of Illinois Urbana-Champaign developed a generative AI framework designed to support more personalized and equitable mental health care. The system creates realistic simulations of patient care journeys, helping clinicians better understand barriers to treatment, improve cultural competence, and develop tailored interventions. In a proof-of-concept study, the AI generated a personalized treatment plan for a fictional young Black man with depression by combining patient-specific information with established evidence-based care models.
According to the researchers, the framework offers a privacy-safe environment for training clinicians and testing interventions without using real patient data. The team also explored how AI could help address disparities in mental health access by accurately representing cultural and systemic barriers. However, they emphasize that AI cannot yet fully capture the complexity of real clinical encounters and should complement, rather than replace, professional judgment.
