JAMA study finds clinicians use OpenEvidence beyond their medical specialties

Clinicians are turning to medical AI for questions beyond their own specialties, often to explore treatment options. A study published October 15 in JAMA examined OpenEvidence use by 950,000 U.S. clinicians, offering a large-scale view of where the tool fits into clinical work—not just how it performs on a medical test.

Researchers led by Harvard Medical School and Brigham and Women’s Hospital analyzed two and a half years of use by physicians, physician assistants and nurse practitioners. Together, those practitioners submitted 270 million queries. The study follows one AI-enabled clinical information platform, rather than measuring clinicians’ use of every AI tool available to them.

Treatment questions outweigh diagnostic queries

To examine what clinicians wanted help with, the team used a large language model to classify the clinical purpose and topic of a random sample of 38.8 million queries. That method gave the researchers a way to sort a substantial portion of the platform’s activity by the medical subjects and tasks clinicians brought to it.

Clinical decision-making accounted for 51.9% of queries, particularly treatment and management. Diagnosis accounted for 13.9%. The findings place treatment choices ahead of diagnostic tasks in this dataset: clinicians were frequently asking about how to care for patients, rather than primarily asking the system to identify a disease.

Those questions often crossed specialty boundaries. Surgeons, for example, frequently asked about subjects other than surgery. Corresponding author Anupam Jena highlighted both the scale of use and clinicians’ tendency to ask about areas where they lacked specific expertise as the study’s two key findings.

Specialties differ sharply in their use

Physicians made up more than half of the study’s users: 556,200 in total. They submitted an average of 37 queries per month of use. Early-career physicians and doctors in oncology, hematology, internal medicine and family medicine contributed a higher-than-average share of queries, showing that activity was not evenly distributed across career stages or disciplines.

The researchers also compared specialties’ shares of queries with their shares of the physician workforce. Oncology and hematology specialists accounted for 3.8% of queries while representing 1.9% of that workforce. Internal medicine doctors accounted for 27.3% of physician queries, compared with 14.5% of the physician workforce.

Radiology, orthopedics and ophthalmology showed the opposite pattern, contributing less than their workforce shares. The comparison distinguishes the sheer size of a specialty from how much platform activity it generates. A large query count alone would not show whether a field used OpenEvidence more heavily relative to its representation in medicine.

A usage study, not proof of better care

The study addresses who uses the platform and what they ask it. It does not answer whether that use improves clinical care. The researchers identified accuracy and effects on patient care as areas for further study. That distinction is especially important here: asking a treatment question is not itself evidence that the answer helped a patient.

The study’s disclosures also matter when weighing the findings. OpenEvidence supplied the de-identified usage data but did not fund the research; three co-authors were company employees. Funding came from the National Institutes of Health’s National Institute of Biomedical Imaging and Bioengineering. The work therefore combines federal research support with company-provided data and company-affiliated authors.