A US healthcare organisation has blocked access to an AI-powered clinical decision support tool after discovering that a physician had entered information from more than 100 patient records into the platform. While some hospitals are restricting access to such applications, others are integrating them into electronic health records (EHRs). The contrasting approaches highlight the challenges healthcare organisations face in balancing AI adoption, patient privacy and clinical safety.
At Monument Health, which operates in South Dakota and Wyoming, the use of OpenEvidence prompted concerns about the handling of sensitive patient information. According to STAT, the organisation's chief information officer, Patrick Woodard, faced objections after access to the platform was restricted.
The decision followed the discovery that a physician had used OpenEvidence to support clinical decisions involving information from more than 100 patient records. The incident illustrates a broader challenge for healthcare organisations. Clinicians can independently adopt AI applications to support their daily work, while hospitals must ensure that the processing of patient information complies with applicable privacy, security and clinical governance requirements.
Growing use of clinical AI
OpenEvidence is an AI-powered platform designed to retrieve and synthesise medical evidence in response to clinical questions. Unlike general-purpose chatbots, the application specifically targets healthcare professionals seeking information to support clinical decision-making. A study published in JAMA examined 271 million queries submitted to OpenEvidence by 952,008 US clinicians between January 2024 and June 2026. The users included 556,200 physicians. Among physicians, questions concerning treatment and patient management accounted for 33.1 percent of queries, compared with 13.9 percent related to diagnosis.
These findings provide insight into how clinicians use the technology. However, the study did not establish whether the AI-generated answers were accurate, how frequently clinicians acted on the recommendations or whether their use improved patient outcomes. The proportion of treatment-related queries nevertheless demonstrates that clinicians are consulting the platform for questions directly related to patient management.
Different approaches
While Monument Health restricted access to OpenEvidence, other US healthcare organisations have chosen to integrate the platform into their clinical information systems. In October, Northwestern Medicine announced an integration with the Epic electronic health record system, allowing authorised clinical teams to access OpenEvidence within their existing workflows. NYU Langone Health subsequently announced a similar integration intended to combine access to medical evidence with relevant patient context.
Such integrations offer healthcare organisations opportunities to manage access and establish controls within existing clinical workflows. However, integrating an AI application into an EHR does not automatically establish its clinical safety or guarantee that patient information is adequately protected. Hospitals must still assess how information is processed, which data are shared with external systems and what safeguards apply.
Privacy and clinical accountability
The use of patient information in AI applications raises questions about privacy, data security and organisational responsibility. In the United States, compliance with the Health Insurance Portability and Accountability Act (HIPAA) is an important consideration. However, assurances from an AI provider about HIPAA compliance do not automatically mean that individual clinicians are authorised to upload protected patient information.
Healthcare organisations must evaluate the contractual arrangements, security measures and operational conditions governing each application. Clinical risks require separate consideration. AI-generated responses may contain inaccuracies, omit relevant information or provide recommendations that are inappropriate for an individual patient.
Organisations therefore need to establish how AI outputs are evaluated, how clinicians can verify the underlying evidence and who remains responsible for decisions made using the technology.
Governance must keep pace with adoption
The challenges are not limited to the United States. European healthcare organisations must assess the processing of personal health information under the General Data Protection Regulation (GDPR). Depending on the intended use and characteristics of an AI application, medical device legislation and requirements under the European AI Act may also apply.
Restricting access to an AI tool can address immediate concerns about unauthorised data processing, but it does not necessarily reduce clinicians' demand for rapid access to medical evidence. Conversely, making an application available through an approved clinical system does not remove the need for independent evaluation, clear responsibilities and ongoing oversight.
The contrasting approaches taken by US hospitals underline the importance of involving healthcare professionals, information security specialists and clinical governance teams in decisions about AI implementation. As AI tools become more widely used in clinical practice, healthcare organisations face the task of establishing conditions under which these technologies can be used responsibly. That requires attention not only to the capabilities of the technology, but also to patient privacy, clinical validation and accountability.
Add ICT&health on Google
Show more content from ICT&health in Google Search.
