Got a burning medical question? Your doctor may be using artificial intelligence to do this.

Dr. Nicholas Gavin, an emergency medicine physician at Mount Sinai in New York, was working overnight last summer when a patient came in with mysterious symptoms. Within seconds, his three junior colleagues—two medical students and one resident—were consulting OpenEvidence, a free AI app for doctors.

Dr. Gavin soon discovered that they were far from outliers. A third of Mount Sinai’s 9,000 doctors were already regular users of OpenEvidence, health system executives found at a meeting with the startup’s leaders last year.

“That was an ‘aha’ moment for our leadership,” said Dr. Gavin, who is also the system’s chief clinical innovation officer.

OpenEvidence’s AI app, essentially a chatbot for medicine, has become a viral hit among doctors. Talk to a doctor, and chances are he or she will use the app to ask specific medical questions or bounce ideas off in a diagnostic dialogue.

More than half of the country’s doctors are regular users. Last month, they used it for 30 million inquiries and consultations, almost double the volume from six months earlier, according to the start-up. Separate last year’s survey of 1,000 doctors found that 45 percent of them used the app, nearly triple the percentage who used ChatGPT, according to Offcall, a career information service for physicians.

That growth propelled the startup to a $12 billion valuation in January, up from $3.5 billion last July.

But the rapid adoption of the app by doctors since its launch in 2024 — one of the few AI-enhanced programs on the market trying to recruit doctors — has raised concerns about how and when the technology should be used in life-and-death situations. In a high-stakes field like medicine, health care systems navigate the vexed issues of patient privacy, security, and trust, as well as the limitations of technology itself.

“It’s not an oracle, it’s a tool,” said Daniel Nadler, founder and CEO of OpenEvidence. “Knowledge and knowledge still matter.”

The doctor’s office has been the target of computer-aided decision-making for decades, with very limited success until recent AI advances.

The first wave of artificial intelligence in medicine focused on alleviating the heavy burden of documentation that contributes to physician burnout by using transcriptions and summaries of patient visits, called AI scribe software. The second wave, which is underway, aims to use artificial intelligence to assist doctors with reliable information and advice to aid in diagnosis and treatment at the patient’s bedside.

Competition has intensified in recent months. UpToDatea popular legacy electronic reference for doctors, has transformed its service into an artificial intelligence with a chatbot interface. proximity, an online professional network for doctors, has bought an AI startup that mines medical literature and generates summaries. Shortenfast-growing AI maker scribe adds decision support tools. And last month he introduced OpenAI ChatGPT for doctors.

OpenEvidence became a pioneer in part because it used exclusively medical journals and other high-quality research as data to train its AI models. Doctors can ask the app specific questions or enter patient characteristics and symptoms and ask for possible explanations. The app complies with federal law protecting patient health information, and doctors are told not to enter any personally identifiable information.

OpenEvidence responds by summarizing the most likely diagnoses and then offers additional “top diagnoses not to miss.” Each includes links to research articles that inform the summary.

“AI is solving some of the problems that have long plagued the practice of medicine,” said Dr. Raja-Elie Abdulnour, director of clinical innovation at NEJM Group, which publishes The New England Journal of Medicine. “Those tools didn’t exist before, and that’s why people are so excited about them now.”

Medical experts agree that early enthusiasm should be tempered with a large dose of caution. The research to date into the advantages and disadvantages of artificial intelligence in medicine is definitely mixed.

AI has passed standard licensing exams and outperformed human doctors in diagnosing certain cases. However, AI has also stumbled by failing to accurately summarize research papers or providing poor answers to diagnostic questions. And they won’t replace humans anytime soon.

“The potential for AI is great, but we’re not there yet,” said Dr. Eric Topol, cardiologist and executive vice president of Scripps Research in San Diego. “It hasn’t really been tested and demonstrated in the messy real world of medicine.”

Dr. Topol is a co-author of a recent paper, “The Illusion of Readiness in Health AI,” which found “significant competency gaps” in the capabilities of large artificial intelligence systems when applied to healthcare.

Evaluations so far have largely focused on the performance of so-called large language models from big tech companies like OpenAI and Google, which are trained on data on the open internet.

OpenEvidencefounded in 2022, took a more targeted approach. The bet was that smaller AI software models trained on highly specialized data could outperform giant models in a specific, information-rich field like medicine. The start-up first trained its software on publicly available medical data from sources such as the government’s National Library of Medicine.

After that, the company entered into content licensing agreements with The New England Journal of Medicine, The Journal of the American Medical Association and other publishers of peer-reviewed medical literature.

OpenEvidence is available to all government-certified physicians in America as a free downloadable application.

“We treated physicians like consumers,” Mr. Nadler said. Users are shown ads, many from pharmaceutical companies, during the roughly five seconds they wait for the AI ​​to respond. Doctors receive ads for only 5 percent of their questions, the company said.

Avoiding the traditional gatekeepers of hospital technology departments has created some problems. OpenEvidence relied on workplace behavior known as “shadow AI,” where workers use such tools without their employers’ knowledge or oversight.

Some healthcare systems are now focusing on introducing OpenEvidence into the institutional realm. Mount Sinai announced in March that it would provide a link to OpenEvidence directly from a patient’s electronic health record.

However, the contract does not give the start-up access to the health center’s patient data. That integration could come later, said Dr. Gavin, but only after rigorous tests and checks.

Protecting patient privacy and safety will be “paramount,” he said, adding that “we’re not going to just throw patient data over the wall of a private company.”

Doctors in smaller practices across the country, especially in rural areas, say technology has won them over.

In Corinth, Miss, with Dr. Ben Long considers himself an AI skeptic. But he was assured that OpenEvidence only generates answers based on high-quality, peer-reviewed information.

Dr. Long initially used it mainly as a reference tool that asked substantive questions. But now he sees the app as more of an “advisor, a thought partner” with whom he has a dialogue, he said.

“AI makes you think more deeply about your own thinking, it challenges your assumptions and why you might be wrong,” said Dr. Long.

Artificial intelligence can also enable doctors to use expertise that would normally belong to the realm of specialists.

Dr. Barbara Creighton often diagnoses and treats complex cases at Community Hospital in Fairbanks, Alaska. They can include multiple conditions and failing organs. In a large medical center, a team of specialists may be consulted – for example, an infectious disease specialist, a pulmonologist and a gastroenterologist.

Dr. Creighton’s little hospital is not so richly staffed. He has an agreement with a large medical center to pay for professional consultations. It now increasingly relies on OpenEvidence to answer many questions, saving time and money at the hospital.

“It’s like having a lot of specialists in your pocket,” said Dr. Creighton.

At Mount Sinai Dr. Gavin said he sees AI technology as a powerful tool to help realize the promise of precision medicine with treatments tailored to individuals.

Progress will require “a patchwork of solutions” from hospitals, medical schools and private companies, he said. Whether OpenEvidence thrives and plays a role in this long-term future remains to be seen.

“But it represents a step in that direction,” said Dr. Gavin.