By MATTHEW HOLT

Okay, I can’t do it any longer. As much as I tried to resist, it is time to write about ambient scribing. But I’m going to do it in a slightly odd way

If you have met me, you know that I have a strange English-American accent, and I speak in a garbled manner. Yet I’m using the inbuilt voice recognition that Google supplies to write this story now.

Side note: I dictated this whole thing on my phone while watching my kids water polo game, which has a fair amount of background noise. And I think you’ll be modestly amused about how terrible the original transcript was. But then I put that entire mess of a text  into ChatGPT and told it to fix the mistakes. it did an incredible job and the output required surprisingly little editing.

Now, it’s not perfect, but it’s a lot better than it used to be, and that is due to a couple of things. One is the vast improvement in acoustic recording, and the second is the combination of Natural Language Processing and artificial intelligence.

Which brings us to ambient listening now. It’s very common in all the applications we use in business, like Zoom and others like transcript creation from videos on Youtube. Of course, we have had something similar in the medical business for many years, particularly in terms of radiology and voice recognition. It has only been in the last few years that transcribing the toughest job of all–the clinical encounter–has gotten easier.

The problem is that doctors and other professionals are forced to write up the notes and history of all that has happened with their patients. The introduction of electronic medical records made this a major pain point. Doctors used to take notes mostly in shorthand, leaving the abstraction of these notes for coding and billing purposes to be done by some poor sap in the basement of the hospital.

Alternatively in the past, doctors used to dictate and then send tapes or voice files off to parts unknown, but then would have to get those notes back and put them into the record. Since the 2010s, when most American health care moved towards using  electronic records, most clinicians have had to type their notes. And this was a big problem for many of them. It has led to a lot of grumpy doctors not only typing in the exam room and ignoring their patients, but also having to type up their notes later in the day. And of course, that’s a major contributor to burnout.

To some extent, the issue of having to type has been mitigated by medical scribes–actual human beings wandering around behind doctors pushing a laptop on wheels and typing up everything that was said by doctors and their patients. And there have been other experiments. Augmedix started off using Google Glass, allowing scribes in remote locations like Bangladesh to listen and type directly into the EMR.

But the real breakthrough has been in the last few years. Companies like Suki, Abridge, and the late Robin started to promise doctors that they could capture the ambient conversation and turn it into proper SOAP notes. The biggest splash was made by the biggest dictation company, Nuance, which in the middle of this transformation got bought by one of the tech titans, Microsoft. Six years ago, they had a demonstration at HIMSS showing that ambient scribing technology was viable. I attended it, and I’m pretty sure that it was faked. Five years ago, I also used Abridge’s tool to try to capture a conversation I had with my doctor — at that time, they were offering a consumer-facing tool – and it was pretty dreadful.

Fast forward to today, and there are a bunch of companies with what seem to be really very good products.

Nuance’s DAX is in relatively wide use. Abridge has refocused itself on clinicians and has excellent reviews, (you can see my interview and demo with CEO Shiv Rao here) and Nabla has just published a really compelling review from its first big rollout with Kaiser Permanente, Northern California in the NEJM no less. (FD I am an advisor to Nabla although not involved in its KP work). And others like DeepScribe, Ambience, Augmedix and even newcomers Innovaccer and Sudoh.ai seem to be good options.

If you take a look at the results of the NEJM published study that was done in Northern California using Nabla’s tool, you’ll see that clinicians have adopted that very quickly, with high marks for both its accuracy, and the ability to deliver a SOAP note and patient summary very quickly. And it has returned a lot of time to the clinician’s day. (Worth noting that independent practice Carbon Health has built its own inhouse ambient scribe and used it on 500K visits so far)

The big gorilla on the EMR side, Epic, has integrated to some extent with Nuance and Abridge, but many of the other companies are both working to integrate with Epic and are inside other EMR competitors – for instance Nextgen is private-labeling Nabla. At the moment, for basically everyone integration really just means getting the note summary into the notes section of the EMR.

But there is definitely more to come. For many years, NLP companies like Apixio, Talix, Health Equity and more (all seemingly bought by Edifecs) have been working on EMR notes to aid coders in billing, and it’s an easy leap to assume that will happen more and more with ambient scribing. And of course, the same thing is going to be true for clinical decision support and pretty soon integration with orders and workflow. In other words, when a doctor says to a patient, “We are going to start you on this new drug,” not only will it appear in the SOAP note, but the prescription or the lab order will just be magically done.

But is it reasonable to suppose that we are just paving the cowpath here? Ambient scribing is just making the physician office visit data more accessible. It’s not making it go away, which is what we should be trying to do. But I can’t blame the ambient scribing companies for that. And as I have (at length!) pointed out, we are still stuck in a fee-for-transaction system in which the health services operators in this country make money by doing stuff, writing it up, and charging for it. That is not going away anytime soon.

But given that’s where we are, I think we can still see how the ambient scribing battle will play out. 

Nuance’s DAX has the advantage of a huge client base, but frankly, Nuance has not been an innovative company. One former employee told me that they have never invented anything. And indeed, the DAX system was massively enhanced by the tech Nuance acquired when purchasing a company called Saykara in 2021, some years after that unconvincing demo back at HIMSS 2018.

So innovation matters, but the other issue is the cost of ambient scribing, which in some cases is nearing the cost of a real scribe. Nuance’s DAX, Suki, and even new entries like Sunoh seem to be around the $400 to $600 a month per physician level. Sunoh is offered by eClinicalworks and has some co-ownership with that EMR vendor. What’s amazing is that at the price quoted at HIMSS of $1.25 per encounter the ambient scribing tool would cost a busy family practice doc seeing 25 patients a day as much as the EMR subscription, around $600 a month.

Abridge has been quoted at roughly $250 a month, and Nabla seems to be considerably less expensive, around $120. But realistically, the whole market will have to compress to about that level because the switching costs are going to be very trivial. Right now, with most of them requiring a paste and copy into the EMR, it’s almost zero.

Which then leads to some more technical issues. How good will these systems become? (Noting that they are already very good, according to reviews on the Elion site). And what will happen to the way they store data. Most of them are currently moving the data back to their cloud for processing. But this may not be acceptable for health systems that like to keep data within their firewalls. For what it’s worth, Nabla, being from the EU and very conscious of GDPR, has been pushing the fact that its process stays on the physician’s local machine – although I’m not sure how much difference that makes in the market.

The other technical issue is the reliance on the large LLMs like OpenAI, Google, etc., compared to companies that are using their own LLM. Again, this may just remain a technical issue that no one cares much about. On the other hand, accuracy and lack of anonymization will continue to be a big issue if more generic LLMs are used. Now the fascination with the initial ChatGPT type LLM is wearing off, there’s going to be a lot more concern about how AI is using health care as a whole–particularly its tendency to “hallucinate” or get stuff wrong. That will obviously impact ambient scribing, even if mistakes may not be as serious as perhaps patient diagnosis or treatment suggestions.

So it’s too early to know exactly how this plays out, but it’s not much too early. In some ways, it’s very refreshing to see the speed at which this new technology is being adopted. As it is, the number of American doctors using ambient scribing is probably below 10%. But it’s highly likely that number goes up to 70%+ in very short order.

The problem that it is fixing for doctors is one that has been around for thousands of years and also one that has been particularly acute for the last twenty years or so. It’s almost like we’re in a period where the doctor suffering with having to  type up their notes in Epic–written up so eloquently by Bob Wachter in his book, “The Digital Doctor,”– is going to be a historical artifact that lasted for fifteen years or so. Maybe it’s going to be talked about nostalgically, like those of us who reminisce about having to get online with dial-up modems.

I’m pretty sure that the winners will be apparent in a couple of years, and that somebody, possibly Microsoft, or possibly the investors in big rounds at 2021 style valuations for Abridge or Ambience, may be regretting what happened in a couple of years. Alternatively, one of them may be a monopoly winner that soon starts printing money.

I suspect, though, that ambient scribing will essentially become a close-to-free product for all different types of business and that clinical care will not be much of an exception. That suggests that a company like Anthropic or OpenAI with close connections to the tech titans, Amazon and Microsoft, will end up becoming more of a feature for the tech giants. My guess is that they will be delivering that product for free probably also into much of clinical care, including ambient scribing. Of course, Epic may decide that it wants to do the same thing, which may leave its partners including Microsoft in the lurch.

It’s reasonable to expect that all aspects of life, including education, general business, consumer activity, and more, will find note-taking, summaries, and decision support a natural part of the next round of computing. For instance, anyone who has had a conversation with their contractor when renovating a house would probably love to have the notes, to-dos and agreements automatically recorded. It’ll be a whole new way of “keeping people honest”. Same thing for health care, I suspect.

But to be fair, we are not there yet. My dictation tool took this whole thing while watching a water polo game on Sunday. And I think you’ll be modestly amused about how terrible the original transcript was. But then I put that entire mess of a text  into ChatGPT and told it to fix the mistakes. it did an incredible job and the output required surprisingly little editing.

AI is getting very smart at working on incomplete information, and health care (as well as clinicians and patients) will benefit.

Matthew Holt is the publisher of The Health Care Blog and one upon a time ran the Health 2.0 Conference

A few months back, we admitted a patient we’ll call Mr. Jones to the hospital for a severe gastrointestinal bleed.  We had discharged him two weeks earlier after he had come in with a heart attack and made sure he was on aspirin to prevent future cardiac events.  He dutifully took his aspirin and on the day of the readmission, had a massive bleed.  He made it to the hospital barely alive and an endoscopy in the ICU showed an active bleeding gastric ulcer.  For Mr. Jones, the gastrointestinal bleed, likely brought on by the aspirin, was an “unintended consequence” that almost killed him. Yet no one questioned whether we should have given him aspirin in the first place.  I felt terrible about what had happened but found solace in knowing that while for some patients the risks of aspirin are worse than the benefits, for the general population of people like Mr. Jones, the benefits are clearly worth the side-effects.

We do risk-benefit analyses every day in clinical care, knowing that for some patients, the benefits will be outweighed by the harm.  We try to be thoughtful about who might be hurt or not, but most of the time, we just can’t predict.  So, when the benefits appear to outweigh the risks, we move forward and try to learn from cases like Mr. Jones.

While this kind of risk-benefit analysis is common in clinical practice, it’s unfortunately not how we discuss health policy interventions.  No policy intervention is ever without risks, and it is rare that a new policy will have no side-effects at all.  Yet, every time policymakers put in a new initiative, they sell it as a panacea. Critics, upon finding an unintended consequence, then declare the whole thing a failure.

An excellent example of this is health information technology, a topic that I have blogged about in the past.  Proponents only talk about its benefits, allowing critics to highlight every shortcoming and failure.  Thank goodness I don’t have to deal with proponents and critics like that every time I consider prescribing aspirin to my patients.

From this backdrop comes an important new JAMA study by Dr. Karen Joynt (on which I was senior author).  We examined what happens to patients with an acute myocardial infarction in states that have public reporting of outcomes for patients undergoing percutaneous coronary intervention (PCI, better known as angioplasty and stenting).  Three states have instituted mandatory public reporting of outcomes for this procedure.

Proponents of public reporting argue that transparency will motivate poor performers to improve and provide information that consumers can use to choose the best providers. However, it was not without controversy.  The critics voiced concern that public reporting would lead to denial of care for the sickest patients because doing so could make physicians’ mortality statistics look worse.  Whether public reporting leads to better care, unintended consequences, or both is an empirical question – a question our research team wanted to answer.

So what was the punch line?  We found that public reporting was associated with reduced access to PCI for patients who had a heart attack.  Stated simply, in states with public reporting of angioplasty outcomes, heart attack patients were less likely to receive this life-saving procedure.  This was especially true among the sickest patients (the ones who probably benefit the most).  Even more concerning, there was a trend towards higher mortality for heart attack patients in public reporting states, presumably because they were less likely to receive this life saving therapy (this effect was small and not consistently statistically significant).  This was clearly an unintended consequence, and a bad one at that.

Transparency – making outcomes public so patients have access to information – clearly has its upsides as well.  There’s evidence that public reporting of PCI has led to lower procedure complication rates, presumably because providers are being more careful. There’s evidence from cardiac surgery that public reporting pushed some of the worst performing surgeons to stop doing surgery altogether.  While difficult to measure, making the healthcare system more transparent to patients has its own benefits in terms of increased trust.

The lesson to draw from our paper in JAMA is that no policy, no matter how well intentioned, is without side effects.  Our job is to think about how to minimize those side effects.  We could improve risk-adjustment methods so that cardiologists get more credit for performing PCI on the sickest patients.  We could carve out some exceptions from public reporting, as Massachusetts has been trying to do, as a response to this kind of data.  Or, ultimately, we can hope that over time, physicians get used to public reporting and simply do what is right for their patients.

When aspirin was first tested in heart attack patients nearly two decades ago, large randomized, controlled trials were conducted, giving us precise estimates of both the benefits and harm.  We could make thoughtful clinical decisions about whether it was worth it or not and in whom.  Most health policy interventions are not tested rigorously before they are implemented.  That’s why we need robust health policy research after policies go into effect.  We need these studies not to discover whether there are any unintended consequences – there usually are – but rather, to quantify the benefit and the downsides so that we as a society can make thoughtful decisions about whether the policies are worth it or not.  Despite the fact that Mr. Jones had a horrible complication from aspirin, I will continue to prescribe it to my heart attack patients.  Despite the fact that transparency seems to reduce access to PCI for critically ill patients, I will continue to advocate for it in healthcare.  The question we now need to ask is how can we do it in ways that lets us maximize the benefit and minimize the harm.

Ashish Jha, MD, MPH is the C. Boyden Gray Associate Professor of Health Policy and Management at the Harvard School of Public Health. He blogs at An Ounce of Evidence.